Igaware Anti-Spam Filtering has been engineered with the overidding aim of ensuring legitatimate emails get delivered.
Igaware Anti-Spam Filtering ensures the highest possible spam detection rates, while avoiding the problems found with many anti-spam solutions; delayed email, loss of legitimate email (false positives).
The only method effective in blocking 100% of spam email, is to disconnect your Internet connection. This isn't very practical!
Igaware Anti-Spam Filtering is the next best thing, with anti-spam detection rates possible of 99.9%. This level of success is achieved using a unique combination of anti-spam technologies, continuous automatic updates, and customer specific anti-spam optimisation.
This guide looks at how Igaware Anti-Spam Filtering works, and how to configure and optimise it.
High Anti-Spam Detection Today
Spammers want you to receive their emails, and they go to great lengths to ensure you do. They are very knowledgeable about anti-spam techniques, and will attempt to bypass anti-spam defences by sending their spam emails in a variety of different ways. It is, however, very difficult to avoid all anti-spam measures, which is why Igaware anti-spam uses a combination of the most advanced anti-spam technologies available. Spammers may avoid one method of anti-spam detection, but rarely all of them.
High Anti-Spam Detection Tomorrow
Spammers are constantly finding new ways to avoid anti-spam detection, which is why Igaware anti-spam is continuously being developed, added to, updated and upgraded. Automatic updates ensure your Igaware Anti-Spam system always has the latest anti-spam defences in place, to ensure detection rates remain high and accurate.
No Email Delays
Igaware advanced anti-spam techniques can identify and block spam emails before they are downloaded, thereby protecting your bandwidth and local IT resources. Igaware anti-spam ensures emails get delivered without delay.
Legitimate Emails Get Through
Unwanted email is called spam, and legitimate email is called ham. But sorting spam from ham can be very difficult as many of your customers and suppliers will be sending emails in ways which will give their emails all the characteristics of spam e.g. in overly complex html format, via an smtp server with a poor reputation, they haven't created appropriate spf records etc. Ensuring sending a good SMTP reputation is a complex task few have got to grips with fully, which is why overly zealous (simplistic) anti-spam systems used by will block legitimate emails.
Igaware anti-spam performs hundreds of legitmacy tests ensuring emails that may have some spam characteristics are still delivered, but with an appended subject of, for example, ‘Possible Spam’. Igaware anti-spam has been engineered to handle ham that looks like spam, and vica versa, so legitimate emails get through. Users can then make the final decision as to legitimacy, something a computer algorithm will never be able to do with 100% accuracy.
Igaware anti-spam scores emails by submitting them to extensive anti-spam tests. You can configure the anti-spam system how to handle email, according to its cumulastive anti-spam score; deliver to recipient as it is, deliver with an appended subject line e.g. 'Possible Spam', delete it or quarantine it. The effectiveness of Igaware's anti-spam scoring approach means you never really need to use a quarantine. This which means you get all your ham straight away, and don't have to spend time looking through a folder full of spam.
No More Non-Delivered Receipt (NDR) Spam Emails
Spammers can forge the sending email address they use. If it's yours, then you could receive hundreds of emails where the original email was rejected or could not be delivered. This is called NDR Spam. Igaware anti-spam can block this type of spam, using email finger printing/watermarking. Igaware anti-spam finger prints legitimate email that you send, and checks for its presence in any emails 'bounced' back. If there's no finger print, it doesn't get through.
You’re in Control
Igaware anti-spam is not a one size fits all anti-spam solution. While it is delivered with default anti-spam settings that will provide excellent anti-spam filtering, these anti-spam settings can be tailored for your organisation. New anti-spam rules can be developed, and existing rules can be ‘tweaked’ with the direct assistance of Igaware’s anti-spam specialists.
Reporting
Email reports provide you with detailed statistics on spam detection, and detail every spam message detected and why it was marked as spam.
Let’s say you compose an email to fred@acme.com. When you press send, the email will be sent to an outgoing SMTP email server, most likely your internal email server. This will then pass the email onto your ISP’s (Internet Service Provider) outgoing SMTP email server. The ISP’s SMTP server then looks at who the message is for, specifically it examines the recipient domain, in this case acme.com. This domain is then looked up, via DNS (domain name service), to determine the MX record (which server handles incoming email for acme.com). The MX record will be something like mail.acme.com. This is where the email will be sent.
The ISP's outgoing SMTP server will start a conversation with the incoming email mail.acme.com. This conversation initially starts with the outgoing email server telling the incoming server who they are, where the email is from, and who it's for. If the message is accepted, it gets delivered.
If an Igaware Anti-Spam system is installed in front of the incoming email server, then the sending SMTP server will, instead, start its conversation with the Igaware Anti-Spam system.
During this conversation, the Igaware Anti-Spam system analyses what is said and, if it's not happy, can reject the email before it has been delivered. The majority of spam is rejected before the sending SMTP server has a chance to send it. This is VERY important as it prevents your Internet bandwidth being eaten up by spam being downloaded. The initial SMTP conversations involve tiny packets of data called headers, and it is based on this that the vast majority of spam can be identified and rejected. This means that a high volume of spam can be filtered without delays to legitimate emails, and loss of bandwidth.
(Note: Hosted Anti-Spam providers often cite the fact that they filter out spam before it reaches your network, thus saving your bandwidth. This is misleading, as a good anti-spam solution doesn't download spam in the first place.
Once an email is accepted for delivery, it undergoes additional anti-spam tests. For each anti-spam test an email fails, it accrues an anti-spam score. The results of each anti-spam test are added togethe,r and a total anti-spam score is then used to determine what happens to that email. Igaware anti-spam allows you to specify how emails are handled, according to their anti-spam score.
By default, Igaware Anti-Spam will deliver emails with an anti-spam score above 3 (low anti-spam score) but below 6 (high anti-spam score), appending the subject with the words ‘Possible Spam Score x’, where x is the actual anti-spam score. Emails with an anti-spam score of 6 or over are deleted. You can change the anti-spam low and high scores, and specify the anti-spam actions taken, including delete, forward to user and send to a quarantine.
This scoring system ensures that legitimate emails get delivered, even when they may trigger some anti-spam rules. This means false positives are avoided.
Only by using a combination of advanced anti-spam techniques, can spam detection be successful. Here are some of the anti-spam techniques used:
Header & Body Test
Igaware Anti-Spam comes with a large set of rules (several thousand), which are applied to determine whether an email is spam or ham. Anti-spam rules have been created using regular expressions that are matched against the body, or header fields, of the message. Each anti-spam rule has a default anti-spam score that is applied to an email, when triggered.
Bayesian filtering
Bayesian filtering uses mathematical probability, to calculate if an email is spam. It analyses the frequency of words found in emails. Particular words have particular probabilities of occurring in spam email and in legitimate email. For instance, most email users will frequently encounter the word "Viagra", in spam email, but will seldom see it in legitimate email. The anti-spam filter doesn't know these probabilities in advance, and must first be trained so it can build them up. The Igaware Bayesian filter has been continuously trained, over 10 years, to correctly identify spam.
Manual address whitelist/blacklist
You can add domains into a whitelist to bypass anti-spam checking, or into a blacklist to block emails.
Collaborative spam identification databases (e.g. DCC, Pyzor, Razor2)
Igaware Anti-Spam uses a number different Hash-based Network Tests, which compare fingerprints of emails against those of previously-seen spam messages (seen globally).
DNS Blocklists (DNSBLs)
Igaware Anti-Spam identifies and anti-spam scores messages, which have been sent from a site listed on one or more DNSBLs
Character sets and locales
Anti-spam rules that check emails for their country of origin and format.
Greylisting
Greylisting will "temporarily reject" any email from a sender it does't recognise. If the email is legitimate, the originating server will try to send it later, at which time the Igaware box will accept it. If the mail is from a spammer, it will probably not be retried as spammers tend to use mail agents, as opposed to proper SMTP mail servers to send spam.
Note: Greylisting requires a single MX record to be set, for each email domain that feeds mail direct to your Igaware box. As it cannot be assumed that this is the case, Greylisting is, by default, disabled. See Greylisting in the next chapter, to find out how to setup an MX record.
Finger Printing
Outgoing emails can be given a unique 'finger print' so, that if emails are rejected or bounced by the recipient, only emails you actually sent are returned. This stops NDR Spam where a spammer has forged your address as the sender. Any such spam emails, that are bounced back to the sender (you), are blocked, as they are seen as not having originated from you in the first place.
The following, explains the recommended configuration for Igaware Anti-Spam. It does not explain every anti-spam option, as these can be seen when accessing the administration interface, along with online help.
The Igaware Anti-Spam system is, by default, already configured. You may wish to change/enable options, depending on your email setup and requirements.
This guide assumes that the Igaware anti-spam appliance has already been installed on your network to receive email. To get the most out of anti-spam, we strongly recommend you receive email via SMTP, by setting up the MX record of your domain to send email to your Igaware box.
To setup an MX record, find out the public IP address that resolves your Igaware Anti-Spam appliance. Next, setup an A Name record, such as mail.yourdomain.co.uk, to resolve this public IP. Finally, set the MX record to mail.yourdomain.co.uk.
Note: You must set the MX record to a domain, not an IP address (a common mistake). There MUST only be one MX record set if you are going to take advantage of Greylisting as an anti-spam measure.
If you collect your email from an ISP, using POP3 or IMAP, the limitation is that all emails must be downloaded and filtered. If your mail box at the ISP has been inundated with Spam, this can cause mail delays. There are no such problems with SMTP delivery. SMTP is the way forward.
To get started, login to the Igaware web admin interface. If you are not familiar with this, we recommend you first read the Igaware User Guide
In the Igaware interface, go to Servers=>Email=>Email Filtering=>General
Ensure that Filtering Enabled is ticked.
Select Anti-Spam=>Settings and ensure anti-spam it is enabled
This is where you set how emails, with different anti-spam scores, are handled. We recommend that you set the low anti-spam score to 3, and the high anti-spam score to 9. Set it that low score spam is delivered to recipients, and high score is deleted.
Select Greylisting, and enable it. IMPORTANT: Only enable this, if you receive your email via SMTP, and there is only one MX record set for your domain.
If you are using a separate exchange server, then go to Servers=>Email=>SMTP Forwarding and, in the forwarding configuration page, ensure ‘Recipient Address Verification’ is ticked. This option will then ensure only emails for known users are forwarded to your exchange server.
That’s it! You have now configured Igaware Anti-Spam. Now, as you use the system you will get feedback from users, as well as detailed email reports about how successful the anti-spam configuration is working. The following section, looks at optimising your anti-spam configuration.
Anti-spam optimisation relies on feedback from users, the use of email reports, and contacting our anti-spam specialists who can recommend configuration changes as necessary.
Here are issues that may occur, and how you can optimise anti-spam systems, to get the best performance from Igaware Anti-Spam.
John says he is getting lots of Spam
Firstly, look in the email reports for John (Activity Reports=>Email Usage Report).This will show details of how many emails John has had, how many were low score spam, and how many were high score spam.
If John is receiving 4 or 5 emails a day, with a low score, this is perfectly normal with the scoring system we use. The scoring system ensures legitimate emails get through. The overhead involved in manually checking 4 or 5 messages a day, is tiny compared to the potential cost of losing an important email.
If spam is getting through that's not getting detected, then please send the email, plus headers (see below), to support@igaware.com, so we can investigate. If necessary, we can tweak anti-spam rules. It may be the case that email, which appears to be spam, is in fact from a legitimate mailing list that the user has forgotten they joined. Emails from a commercial mailing list, will normally contain an unsubscribe option.
To get headers in Outlook, open the email and select View=>Options and copy the Internet Headers (Highlight the text in the Internet Headers box and press Ctrl C on your keyboard to copy)
Users should also be made aware that it is impossible to stop 100% of Spam. Having said that, they should expect very little spam to get through, especially with Igaware Anti-Spam detection running at 99.9% detection success.
A legitimate email has been marked as Spam
You can whitelist email domains you trust, by going to Anti-Spam=>Settings
To see why an email has been marked as spam, look in the headers for the line starting with; X-1055692938-MailScanner-SpamCheck:
This line will show the spam score, and which anti-spam rules were triggered. Here’s an example:
X-1055692938-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
score=7.017, required 3, BAYES_99 3.50, HTML_MESSAGE 0.00,
MIME_HTML_ONLY 1.46, RCVD_IN_PBL 0.91, RDNS_NONE 0.10,
SPF_NEUTRAL 0.69, URI_HEX 0.37)
In this line you can see that the anti-spam score was 7, and one of the anti-spam rules triggered was one of the Bayesian tests (BAYES_99), which attributed a score of 3.5.
You can send the headers to Igaware, and we can then report back as to what action the sender should take. If they are being marked as spam, then it is very likely that their emails will be having delivery problems to other recipients, and your feedback can help them.
An email has not been received
It is possible, that a sender has a serious problem with their email system, causing their emails to receive a high anti-spam score. The first thing to check is the email activity reports, to see if the email has been received. You can also download the mail logs in System=>Tools=>Log Viewer. The logs show every incoming SMTP conversation, allowing you to search for the sender’s domain.
If an email had not been seen in either of these logs, then this is an issue with the sender’s email transport system, and something they will need to investigate. They may well have received an NDR (non delivered receipt). The NDR will tell them why their message was not delivered.
If an email has been given a high anti-spam score, then check the headers and send them to us, so we can see what is going on. Chances are, the sender has a serious problem. In the meantime, you can white list the sender’s domain.
Note: Emails may be blocked if they contain viruses, or dangerous attachments. The email activity reports will show you if an email has been blocked, and why, enabling you to take appropriate action.
Emails have been delayed
If an email has taken a significant time to reach you, the easiest way to diagnose where the delay occurred, is to look in the email headers. In here, you can see when it was sent, and the time it was passed on along the way. Igaware Anti-Spam has been engineered to avoid email delays. The headers will show you where the delay occurred.
Anti-Spam rules are found in Anti-Spam=>Spam Rules/ Score. Each anti-spam rule has a score, which it applies to an email that triggers it. Scores for existing anti-spam rules can be changed, and new rules manually added, but we recommend that this is only done by an Igaware engineer, to resolve a specific Spam detection issue that has been raised. Anti-spam rules are continuously updated automatically, so you don’t need to become an anti-spam expert. In general, once the Igaware anti-spam system has been installed, you can forget about it, as it looks after itself.
Igaware can help with any aspect of configuring and optimising the anti-spam system. For help, please email support@igaware.com.
Learn all about Spam on the web at http://en.wikipedia.org/wiki/Spam_(electronic)
To learn more about Anti-Spam call us on 0191 303 6177 or email support@igaware.com