I believe IE loads faster because it is integrated into windows, which is also not good for security.
The more extensions you have the longer it will take Firefox to load, I've read that some people with 50 or so extensions can cause Firefox to take about 4 seconds to load, I defrag regularly have only about 3 extensions so the longer loading time is barely noticeable for me.