Question: Any idea what the difference between: info.magnolia.rest.RestDispatcherServlet and info.magnolia.rest.RestJerseyDispatcherServlet is?

Answer: RESTEasy and Jersey impls are meant to be on feature parity.
Magnolia is typically deployed on Tomcat, which has no REST "engine", so we provide RESTEasy OOTB (but you can also provide Jersey instead). Only for certain application servers, could be WebSphere iirc, Jersey was the default REST (JAX-RS) impl, so one could exclude the whole RESTEasy libs and just rely on the ootb REST support.

See MGNLREST-82: Introduce RestJerseyDispatcherServlet to handle endpoint registration with jersey