There's one huge jump from 12GB to 4GB there...
I suggest something customizable, like Slackware or Debian, that doesn't automatically come with KDE or GNOME, and has a light text installer. Fluxbox or FVWM should run comfortably on that. You could try DSL, because I know for certain it will run on 32MB of RAM, but it is a bit skimpy.