Spam is a hot topic and still getting hotter. Fighting with spam is becoming more and more crucial and regular task for IT Pros like us. In order to win the battle finally, by knowing the essential knowledge about the technology will definitely help. In the Windows marketplace, no "one-size-fits-all" antispam solution exists, so by knowing the knowledge also will help to decide what to look for when you're evaluating which type of antispam solution is appropriate for your organization. Here, I listed something technologies and solutions I have learned from my research and some experiences I have got from the battle of the fight during my work. Hopefully, it would give others some ideas and help them out from the pain.
Antispam Solutions
A. Filtering at the client
Many antispam solutions install on a user's PC alongside an email client (e.g., Outlook, Eudora). This type of solution is usually easy to deploy for an individual user, and it has an immediate beneficial effect on the user's productivity. However, it is not suitable for enterprise level. An example of this type of solution is Qurb from:
http://www.qurb.com/
B. Filtering at the email server
This solution deals with spam at the email server end before they arrive in a user's inbox. But they do this at the expense of creating extra work on the servers; you need more queues and processing power. The spam messages still have to travel the length of your email system before they are detected, so there is no beneficial effect outside the server environment. An example of this type of solutions is Microsoft Exchange Inelligent Message Filter at:
http://www.microsoft.com/exchange/downloads/2003/img/
C. Filtering at the Internet Gateway
Many vendors provide spam-filtering solutions that work at the point that email messages enter your organization. Some of them are software products that install on a standard Windows or UNIX server; others are all-in-one appliances that plug into the right place in your network and that you configure and manage through a web interface.
These solutions provide a good level of relief to your email infrastructure. If spam can be dealt with at the gateway, it doesn't clog up your email servers or your internal network infrastructure. Gateway solutions require some reconfiguration of the way email traffic flows into your company and you should be aware that a spam-filtering gateway can become a single point of failure for your email network. An example of this type of solutions is GFI MailEssential from:
http://www.gfi/mes/
D. Filtering outside your network
A filtering solution provided away from your network by a managed service provider (MSP), has the features of the gateway offerings but also has some extra benefits. A simple change to the DNS records for your domain will cause all your email traffic to be delivered by the MSP's resilient servers. Those servers filter the email traffic for spam and only deliver to your gateway the mail you actually want to receive. No unwanted traffic flows through your Internet connection. An example of this type of solutions is Postini Perimeter Manager at:
http://www.postini.com
Spam Detection Technology
The spam detection landscape, which is constantly evolving, has been likened to an arms race. As antispam developers come up with a new technique to recognize and block spam messages, the spammers counter this with new methods of their own to get around the filters.
Content tests
- Keyword checking
It used to be an effectifive measure - messages that contained particular words were easily recognized as spam. But now this simple filters are no longer very effective.
- Bayesian
It happened to be the most effective way of detecting the spams by testing the content. Some solutions, using Bayesian algorithms that analyze the patterns of words found in spam and non-spam messages, are able to predict whether a message is spam based on evidence from all the previous messages they've seen. Another words, it needs to be trained before acting as a good detective.
Connection-level tests
Those methods above rely on being able to analyze the content of an email message. But recently, spam senders have been able to bypass this kind of analysis by sending messages that have little or no spam content. These spam messages may, for example, only contain a URL that loads an advertisement into the message from a remote web server when the receiver views the message.
Fortunately, there are other mechanisms for detecting spam, which rely on the characteristics of how a message arrived at the server, for example, which IP address it came from, what other recipients it was sent to, whether other people have received spam messages recently from the same source, how well the sending server complied with established SMTP protocols, and so on. These connection-level tests can be very effective at detecting spam messages, regardless of whether they have any content that can be analyzed.
- DNS Blacklist
- Header Checking
- SPF (the Sender Policy Framework) - the new technology developed to fight with those spams sent from the forged sending addresses as most of today's spammers spoof email addresses. Visit http://spf.pobox.com for more information.
- Directory Harvesting
A good antispam solution needs to apply these connection-level tests, as well as content tests, to give you good protection. Gateway and MSP solutions that intercept the email traffic on its way to your network, are well placed to cover both angles.
Consideration
When you start thinking about putting the solution in-place to prevent the spam, first of all you have to consider which kind of solution is suitable for your organization. I highly recommend the solution C unless your organization doesn't have proper person to manage it and the message flow is not that high confidential.
Secondly, you have to choose the right software/hardware, a good software/hardware should include the feature like below. Personally, I would vote GFI MailEssential product as the best.
- Auto-whitelist - to minimize the on-going false positives.
- Bayesian - to catch the most content-based spam.
- DNS Blacklist
- Header Checking
- Flexible configuration to tune the filter like learning mode, more setting on each feature listed above, etc.
Thirdly, once you decide which solution is going to be used and which software is going to be in-placed, you have to make a detail plan how to get it implemented seamlessly. A bad plan will turn your whole job down whether you choose the right solution and software or not. Plan it as much detail as you can ahead and do it as much carefully as you can after would definately accomplish your project successfully. Also, every anti-spam software needs to be tuned and trained to achieve their best performance, so be prepared properly would turn down the trouble to the minimum from the beginning.
- Prepare a company customer email list and import it to the whitelist.
- Run the bayesian learning wizard to scan some existing mailboxes' sent folder to raise the number of the legitimate in bayesian database before enabling it.
- Do not set up to delete the spam, instead, set up to forward to a central mailbox and assign a person to monitor it at least for a couple of weeks.
- Adjust the other settings on keyword checking, header checking to make sure it works on your situation.
- Turn on the auto-whitelist.
- Move missed spams to spam folder to improve the database for the future.
- Move false positives to legitimate folder to improve the database for the future.
Finally, I hope we all can win the battle and say "There is no door here for you" to all spammers.
Tips. not every software says they can turn on filter for these users and turn off for the others. However,here is the trick you can accomplish it on most of products. If you want exclude a user from spam filtering, simply enter the email address of the user, and select MIME TO option.