Use a simple email client
The biggest problem is e-mail. E-mail can deliver a virus/trojan program right to your computer that can destroy your computer (or do any number of nasty things) with a single click if your e-mail program is able to automatically do a bunch of other things. For this reason, it is a bad idea to use Outlook Express with Windows. Similarly, if you read your email with a web browser instead of retrieving it to your machine (yahoo, hotmail, etc), using Internet Explorer to do that is a bad idea. Why do you lock your doors at home and in your car? Because the convenience of unlocked doors is not worth the risk of burgulary. The same rule apples here.
Any other email program is safer than Outlook Express, and the simpler it is the better. I recommend the use of Thunderbird these days.