Spam Filter ISP

Blocked | Allowed | Options | Lists Tab | Whitelists
Customize | Bayesian Filter | SPF spam filter | Log Analysis | Requirements

SPAM Filter ISP is configured to be your spam filter gateway, and handles all incoming emails going to the servers listed for your MX records. SPAM Filter ISP can be configured to listen on a specific IP, multiple IPs or all IPs bound to the NIC card. See Configuration Section for more details.

Which Emails Can Be Blocked (Blacklisted) Top^

• Images - New! - Our SPAM Filtering software is able to scan inside images embedded in emails for spam content.
• SFDB Filter - New! All the thousands of SpamFilter installations in the world are networked together to create a huge database of spammer's IP addresses. SpamFilter uses this database, updated in real-time, to block spam.
• MAPS Servers - Our SPAM Filter server checks the IP address initiating the connection. If it is listed in one of its many DNS RBL blacklist servers the connection is refused. 
• SURBL Servers - SpamFilter scans the content of emails for any HTTP links and URLs. Every link found is then tested against one of the many SURBL DNS blacklists available. If present, the connection is refused. 
• Local IP Blacklist - Our SPAM Filter server checks if the remote server's IP address matches an entry in your local IP blacklist file, the email is rejected. 
• Local Domain Blacklist - The SPAM Filter gateway checks if the domain portion in the sender's email address is in your local domain blacklist file, the email is also rejected. 
• Local Country Blacklist - The sender's Country is tested to see if it is in your list of undesired countries. If so, the email is refused. This product includes GeoIP data created by MaxMind, available from http://maxmind.com. 
• Local FROM EMail Blacklist - The sender's email address is checked against your local list of blacklisted email addresses. If present, it is rejected. 
• Local TO EMail Blacklist - The recipient's email address is checked against your local list of blacklisted email addresses. If present, it is rejected.
• Attachment Blocking -  SPAM Filter can check emails for specific attachments or attachment extensions. If found, the email is rejected. 
• Keyword Content Filtering - Our SPAM Filtering software can check email content and subject for specific keyword and/or phrases. If found, the email is rejected.
• Bayesian statistical DNA fingerprinting -  The new v2.x release of SPAM Filter ISP features statistical DNA fingerprinting of incoming emails. This filter is self-learning, continuously analyzing your incoming traffic to improve its accuracy with time. 
• SPF - Sender Policy Framework - SPF fights email address forgery and makes it easier to identify spam, worms, and viruses. Domain owners identify sending mail servers in DNS. SpamFilter ISP verifies the envelope sender address against this information, and can distinguish legitimate mail from spam before any message data is transmitted.
• Honeypot Emails - You can have a list of "honeypot" email addresses. Any email sent to an address in the list will cause the sender's IP to be blacklisted.
• Additional anti-spam tests - SPAM Filter can then optionally check to see if the recipient address has a % sign in it. Many SMTP servers are susceptible to being tricked into relaying with this. Connections can be rejected if the remote server does not have a reverse DNS PTR entry. Spam Filter is able to check if the sender's MX DNS record is valid before accepting email. You can also refuse connections if the remote server attempts more than n RCPT TOs in a single connection or if there are too many spaces in the subject line.

Which Emails Are Allowed (Whitelisted) Top^

• Allowed domains - If the IP passes the DNS tests SPAM Filter then checks the recipient domain. If the domain is listed as a local domain, then the recipient is accepted. This is done to prevent spammers to use SPAM Filter to relay. 
• Excluded IPs - If an IP is blacklisted, but you really need to be able to receive email from that domain anyways, the domain can be added in an exclude list as to allow it to bypass the blacklist rules. 
• Excluded Domains - If an IP is blacklisted but you still wish to receive email from them, the IP can be added to an IP exclude list to allow it to bypass the blacklist rules. 
• Unfiltered Emails - If you have users who do not want to receive filtered emails, they can be accommodated by adding them to a pass-list. EMails addressed to them will bypass all of SPAMFilter rules. 
• Excluded FROM Emails - If you want a sender's email address to be excluded from all filtering rules, you can add it to an exclude list. Our spam filtering software will allow all such emails to be delivered.
• Authorized TO EMails - If you want SPAM Filter to only deliver emails to specific addresses in your domain(s), you can manage such a list here. Please not that if such a list is present, SPAMFilter will not deliver email to an address unless it is present in such a list. Use with care.
• Keyword whitelisting -  You can provide your customers with specific keywords that, if found in the body or subject of emails, will bypass all filtering rules. 

Other Options Top^

• Max concurrent incoming SMTP connections - You can limit the maximum number of concurrent incoming connections here. 
• Max Recipients in single session - Use this setting to limit how many RCPT TO commands can be issued in a single session. 
• Min MAPS matches needed to reject msgs - Sometimes MAPS blacklists can be too strict and list legitimate domains in their blocklists. You can reduce the number of false positive by requiring that more than one single blacklist match is found before rejecting a connection. 
• Max Email Size - Incoming emails can be blocked if they exceed a certain size.
• Process queue every n minutes - Use this setting to control how often SPAMFilter attempts to redeliver the items on hold in the queue directory. 
• Max number of spaces in subject line - Many spam messages contain large number of spaces and tabs, they can be filtered here.
• Bayesian Filter Threshold - Use this slider to control the accuracy of the statistical filter. Incoming emails are assigned a probability of being Spam, ranging from 0% (most likely a valid email) to 100% (most likely Spam). Any emails that have a probability of being spam above the value you set will be rejected. Typical threshold values are in the 99.9% range.
• Days to archive rejected emails - Normally SPAM Filter will reject an email if it considered as spam. You can optionally choose to receive and archive those emails rather than having them lost. The remote server will still receive an error stating that the email was rejected, but you will keep a copy in the quarantine directory for this amount of days. This will allow you to force delivery of legitimate email which could have been filtered. If you enter a 0 in this field quarantine is disabled and email is rejected immediately. 
• Allow % in address - SPAM Filter can then optionally check to see if the recipient address has a % sign in it. Many SMTP servers are susceptible to being tricked into relaying mail with this. Ex. if you are isp.com, then a spammer could try to use joe%yahoo.com@isp.com to relay mail to joe@yahoo.com if your server is vulnerable. 
• Logging - Check this box to enable logging in the log directory. 
• Remember Stats - Check this box to save the email statistics when shutting down SPAMFilter. 
• Disable Connections Grid - The Connections tab will show you in realtime what the various connections on your servers are and what they are doing. If you have a busy site with 500 concurrent connections this list can get pretty crowded and unwanted.... 
• Auto-check for new build - If checked SPAM Filter will connect with our website to see if a new version is available. SPAM Filter will issue a simple GET request to http://logsat.com/SPAMFilter/version.htm to retrieve the version number. Absolutely no data will be sent to us! 
• Tag Spam & Deliver - Allows to tag spam by adding the header "X-SF-SPAM:Y" to email classified as spam. The email is then forwarded to the destination SMTP server. This allows administrators to handle spam as they wish on the back-end.
• Tag Spam in Subject & Deliver - Allows to tag spam by prefixing the word SPAM: in the subject line of emails classified as spam. The email is then forwarded to the destination SMTP server. This allows administrators to handle spam as they wish on the back-end.
• Enable Cached IP Blocking - If an IP address sends more than a certain number of spam emails (3 by default) during a certain time interval (10 minutes by default), then it can be temporarily banned (blacklisted). All further connections from that IP address will be immediately rejected without allowing the sender to transmit any data. This should greatly reduce the load on the server. A banned IP address will be automatically removed from this temporary blacklist after a defined time interval (60 minutes by default).
• Reject if no reverse DNS - SPAM Filter can be configured to reject emails if the remote server does not have a valid reverse DNS PTR entry. 
• Reject if Empty "Mail From" - If this option is checked SPAMFilter will reject all emails with an empty "Mail From" field. Please note that this setting will delete legitimate email, as in email receipt notifications and some error emails. 
• Reject if "Mail From" = "Mail To" - Reject all emails where the sender's email is the same as the recipient's email. Note that this causes problems with users who send emails to themselves using EBay's web interface for example.
• Reject if "From Domain" = "To Domain" - SPAM Filter can reject all email where the sender's domain is the same as the recipient's domain. Usually your users will not go through SPAMFilter when sending emails to themselves, Spammers often use this technique.

Black/Whitelists Tab Top^

• BLACKLISTS MAPS Blacklist servers - SPAM Filter checks the IP address initiating the connection. If it is listed in one of its many DNS blacklists the connection is refused. SPAMFilter can reject connections based on a configurable minimum number of matches. 
• Blacklisted IPs - You can keep a file with additional IPs that you want to blacklist by entering the filename below. If the file does not exist it will be created. The file is reloaded every minute. List individual IP addresses on each line. Use an ending .0 for a Class C wildcard (i.e. 192.12.45.0 to block 192.12.45.1 --> 192.12.45.255). The contents of the file will be loaded in the memo box, allowing you to make changes to the file. 
• Blacklisted Domains - You can keep a file with additional Domains that you want to blacklist (based on the MAIL FROM field) by entering them below below. Enter one domain per line, no wildcards allowed. If the file does not exist it will be created. The file is reloaded every minute. The contents of the file will be loaded in the memo box, allowing you to make changes to the file.
• Blacklisted Emails - If you want to block any particular email addresses, enter them here, one email per line.
• Country Filters - SPAM Filter checks the what country incoming connections are coming from. The current number of connections for each country can be updated by clicking on the Update Stats Now button. Columns can be sorted by clicking on the column header. This will help you in sorting countries and hits so you can determine if there are any countries you do not wish to receive email from.
• Attachment Blocking -  You can block emails that have unwanted attachments. You can keep a file with banned attachments here. check emails for specific attachments or attachment extensions. If the attachment is found, the email is rejected.
• Keywords Filter - You can check email content and subject header for specific keyword and/or phrases. If found, the email is rejected. You can also use Regular Expressions (RegEx). If the keyword file does not exist it will be created. The file is reloaded every minute. The contents of the file will be loaded in the memo box, allowing you to make changes to the file. The rules are as follows:

Whitelists Top^

• Local Domains - SPAMFilter will only deliver email to the domains listed here. If the domain in the RCPT TO email address is listed as a local domain, then the recipient is accepted. This is done to prevent spammers to use SPAMFilter to relay email to third party email addresses/servers. If you need to have any domain listed here forward its destination email to a different server than the default destination server, you can specify so here. You can override the default destination server by appending the forwarding mail server and port to any domain in this list. The syntax should be as follows: DomainName:DestinationServer:DestinationPort - example: logsat.com:mail.netwide.net:25 
• Excluded Domains / IPs - Add here any "MAIL FROM" domains or any IPs from which you want to receive email if they would be blocked by any of your blacklist rules. Enter as many IPs or domains as you wish, one per line. 
• Unfiltered Emails - Any local email address listed here will cause SPAM Filter to bypass all blacklist rules for it. If you have any users who do not want to have their email filtered, enter them here. 
• Keywords Filter - You can check email content and subject header for specific keyword and/or phrases. If found, the email is allowed through the filters. Useful if you want to allow certain customers to send you email without having to place them all in a email address whitelist. The same syntax rules as the blacklist keywords apply.

Customized Items Top^

• Most rejection notices to the remote servers can be customized. In the error string you can embed the following connection-specific parameters:
• %IP% - The IP address of the remote server connecting to SPAMFilter 
• %Domain% - The MAIL FROM domain name of the incoming email attemp 
• %EMailTo% - The recipient of the incoming email attempt 
• %EMailFrom% - The sender's email address

Bayesian Statistical Filtering

The new v2 release of SpamFilter ISP features statistical DNA fingerprinting of incoming emails. The statistical analysis is performed using Bayesian rules. Tokens within incoming emails are scanned and categorized in a corpus file. The content of all new incoming email is fingerprinted and checked against the historical data. If there is a high statistical probability that the email is spam, it is rejected.  The statistical engine kicks in after 5,000 non-spam and 5,000 spam emails have been received (values customizable by editing the SpamFilter.ini file). This is done to build a valid statistical base to use before emails are rejected. During this period of time, it is critical to avoid false positives. If a good email is quarantined, forcing it's redelivery either thru the web interface or the SpamFilter GUI will "teach" SpamFilter that the fingerprint in that email is a "good" one, and the statistical DNA database will adapt itself to it. It is very important initially to check the quarantine often to force delivery of legitimate email that has been blocked by the "regular" filtering rules.

A slider is used to control the accuracy of the statistical filter. Incoming emails are assigned a probability of being Spam, ranging from 0% (most likely a valid email) to 100% (most likely Spam). Any emails that have a probability of being spam above the value you set will be rejected. Typical threshold values are in the 99.9% range.

SPF - Sender Policy Framework

SPF is an open source standard that is emerging as a solution to prevent spammers from using fake email addresses. The following description was taken from the official SPF website at http://spf.pobox.com:

Domains use public records (DNS) to direct requests for different services (web, email, etc.) to the machines that perform those services. All domains already publish email (MX) records to tell the world what machines receive mail for the domain.

SPF works by domains publishing "reverse MX" records to tell the world what machines send mail from the domain. When receiving a message from a domain, the recipient can check those records to make sure mail is coming from where it should be coming from.

With SPF, those "reverse MX" records are easy to publish: one line in DNS is all it takes. Suppose a spammer forges a hotmail.com address and tries to spam you.

He connects from somewhere other than hotmail.When his message is sent, you see MAIL FROM: <forged_address@hotmail.com>, but you don't have to take his word for it.

You can ask Hotmail if the IP address comes from their network.
(In this example) Hotmail publishes an SPF record. That record tells you (your computer) how to find out if the sending machine is allowed to send mail from Hotmail. If Hotmail says they recognize the sending machine, it passes, and you can assume the sender is who they say they are. If the message fails SPF tests, it's a forgery. That's how you can tell it's probably a spammer.

Spam Filter ISP looks up SPF DNS records for all incoming emails. If an SPF record exists, the query results can be any one of the following:

• Pass: the message meets the domain's definition of legitimacy.
• Neutral : the message does not meet a domain's definition of
  legitimacy, but the SPF client MUST proceed as if a domain did not
  publish SPF data. Likely used by domains in transition phase
  who are beginning to adopt SPF.
• Softfail : the message does not meet a domain's strict
  definition of legitimacy, but the domain cannot confidently state
  that the message is a forgery.
• Fail : the message does not meet a domain's definition of
  legitimacy.

If the result is "Pass" the email will pass the SPF filter. Behavior for all the other failing results can be customized by the administrators in the SpamFilter GUI by adjusting the settings in the Settings - SPF Filter tab.

SFDB Filter - SpamFilter Distributed Blacklist

The SFDB filter uses a very powerful resource to stop spam:
The entire global SpamFilter ISP user community.

This latest filter is proving to be one of the most effective and accurate tools in stopping spam.

Anytime SpamFilter ISP blocks an email, the sender's IP address is sent to our centralized SFDB database. This allows the SFDB filter to have access to a huge repository of spammer's IPs, updated in real-time by all the SpamFilter ISP installations in the world. IP addresses from the database are automatically aged and removed from the database within 6-24 hours if they stop sending spam and/or viruses.

The SFDB filter detects spam by checking IP addresses against the SFDB database. The "network reliability" level tells SpamFilter how many different users must have reported a specific IP in order to classify it as spam.
 

Log Analysis & Statistics

Spam Filter ISP log files can be parsed by Sawmill, an excellent log analysis tool. Sawmill generates reports of email traffic by IP, domain, country, sender and recipient, action taken on messages and much more. In the SpamFilter\Database directory you will find the Sawmill plug-in file SpamFilterISP. If your copy of Sawmill 6.5 or higher does not recognize SpamFilter ISP's log format, simply copy that file in the Sawmill\LogAnalysisInfo\LogFormats directory to allow it to read SpamFilter ISP logs.

System Requirements

• Software - Operating System: Spam Filter ISP will run on Microsoft Windows NT4, Windows 2000, Windows XP, Windows 2003.
• Hardware: Spam Filter is very CPU and RAM efficient. Server requirements will depend on the email traffic. For a server handling 20,000 emails/day, a 500MHZ CPU and 512MB of RAM is the minimum recommended. VMWare virtual servers are also supported.
• Optional quarantine database: Microsoft SQL Server 7 and higher, MySQL 4.0 and higher, Oracle 8 and higher, Microsoft Access 2000 and higher.