You are correct, if you plugged the routers into the switch they wouldn't be able to interact. By their nature, switches keep connections completely seperated.
I know you didn't ask for product recommendations, but I think
this is a cheap solution for you. Looks like it has good reviews as well.