The biggest reason I would say NOT TO use IE (or Microsoft's mail programs) is compatibility. 100% compatibility with every exploit designed to take over your Microsoft branded OS. 100% compatibility with ActiveX, the worse idea EVER in client software. Someone could write you a nice ActiveX control to delete your partition table and overwrite random sectors on your hard drive until your computer crashed if they wanted. You couldn't prevent it. Years after Microsoft 'fixed' all the bugs that 'let malicious code execute because a picture popped up in your inbox', they're STILL fixing those bugs!
Just staying away from 'odd' sites isn't enough. People go around cracking web sites and redirecting them to the 'odd sites' that look just like the real ones. Most people dismiss that 'I'm about to install a plug in' warning and let it install anything from any site at any time, rendering whatever security has been engineered into the system useless. Especially since it's 'Whack a key' or mouse 'Muscle memory' buttons people are trained to dismiss with endless repetition, like in Vista.
After all these years, people are still writing nasty things that exploit the standard availability of Microsot's 'Address Book' to flood all of your friends and family and coworkers with X-rated spam in your name.
Mix it up a little. It doesn't matter WHAT browser or email client you use (I use Firefox and gmail's web client), as long as it is NOT Microsoft's.
1. Don't use IE, Outlook, Outlook Express, Microsoft Mail, etc. As I said, 100% compatibility with the vast majority of viruses, worms, spyware and all manner of malware. When criminals write this stuff, they save time and money by developing it for the #1 prevalent combination OS, email and browser. It's called a 'monoculture', and they're always bad. Be the 'mutant' that doesn't die from the latest, nastiest blight because he/she ran something different.
2. Don't let ANY browser remember form entries and passwords. Go into the settings and turn that crap OFF! Remember them yourself. If something does get in, your phone number, mailing address and shopping habits will be wide open (you pecked it into dozens of online shopping forms) and your bank password will be appreciated when they crack your uploaded password cache about five minutes after they get it.
3. You don't need different passwords for every site, but you do need different passwords for every IMPORTANT site. Too many people use the same login/password for everything. Is your PC World password as important as your bank password? What happens if 'PC World' gets hacked and the username/password database is compromised? Nothing much. Of course if someone gets their hands on that list and starts trying those user names and passwords on high value web sites like PayPal, Ebay, major etailers like Amazon that remember your credit card, banks, credit card sites, etc., and you use the same login everywhere, chances are good you'll get boned. Change your important passwords!
Reasons to use Firefox...
Well the main one is a nice little setting in "Edit->Preferences->Privacy->Always clear my private data when I close Firefox", with a button to set what to clear (I clear everything, every time). Turn it on. Unlike IE that has historically made inaccessible logs of everything you do with the browser, Firefox can 'forget' your session after you close Firefox. Turn off 'Remember Passwords', too. Nothing can make stored passwords safe. Nothing.
Why would you want it to 'forget' things? Besides visiting off-color web sites? Besides corporate policies for spying on the company's portable PCs? Besides privacy, in case someone pokes around inside my computer? Because the browser can temporarily 'remember' web sites you visited during your current session, accept cookies, remember forms in case of errors, etc., and then when you close it, it will forget - erase everything.
Another nice feature, yeah Add-Ons. Adblock Plus and Adblock Updater, and Flashblock. The internet is so much nicer with these installed. No banner ads, no animations with noisy sound effects popping up in your face after they choke your browser to a crawl. The first two simply filter typical ad sites. You can pay $50 a year to install a proxy server that blocks ads in IE and stays up to date, or you can get a free, simple plugin that doesn't remain running after you close the browser. The Flashblock one replaces Flash content with a button to load and play Flash content on demand. So you go to a web site with a bunch of flash nonsense on it, you can click the big, middle pane with the content you want, or decide "Oh, it's just full of cartoons and has no useful content", like most car company web sites, but you can find that out right away rather than at the end of a 14MB download of crap. Also, if you go to YouTube or LiveLeak, and you spot a dozen interesting things to watch, you can open them ALL in tabs and play them as you get around to the tabs, rather than have the computer choke when it tries to play a dozen videos all at once. A lot less back-button pressing and scrounging around retracing your steps.
All of these plug-ins are free and also have settings to temporarily or permanently 'ignore' certain sites. Some of the AtomFilms based sites break down if the ad-blocker of any sort blocks an ad. AtomFilms COULD simply serve up the ads themselves as part of the movie clips, but no, they choose to have an external and well-recognized ad service inject ads, and have the content quietly fail to play with no explanation.