My antispam filtering system

• First of all on our MX we have installed sendmail with DNS Black Hole list querying. That is very effective (some 50 contacts per day are refused) to reject known open relays and other exploited sites.
  Unfortunately that seemed to reject most spammers for my colleagues but not for me.
  Note added in summer 2006 The above statement is no longer true, and the procmail rules described here are still active but only on the few residual spam which survives our current MX filtering, based on the sendmail milter of amavisd-new which calls spamassassin.

• Therefore I still continued to use procmail, but with a new set of simplified rules in this order.

  ---

• First I use rc.black to reject unconditionally the e-mail from a few persons. This uses file blacklist. This is more a kill filter (it is not effective against spammers).
  Additionally I set here some variables with the path to daily-stamped folders which will host non-rejected suspect spam, which will be used in later rules.
  Optionally one can add at this stage a filtering against some mailing lists which are no longer in use (like for us the CNR staff list) to divert its mails to somewhere else (e.g. for convenience the suspect spam folders). This can be done nesting other rc files.

• Note added in January 2009 I added at this step rc.received which analyses the Received keywords in the header, and sets a TRUSTEDMX variable if the message is coming through an MX whose IP address is whitelisted in a file mx.analysis.sort.
  This variable is used in some of the other rules to add favourable weigth to some scores.
  Since this can be of general use I provide a separate documentation.

• Then I use rc.html-virus which I won't show. It does three or four things :
  1. computes a boolean (yes|no) variable OLDDOMAIN if the e-mail were addressed to our old domain. My old e-mail address was harvested by spammers probably when I used it for Usenet postings. Thus I receive most if not all spam under the old address.
  2. computes a boolean FAKEADDR, considering "fake" an e-mail where the domain of the sender (i.e. whoever will get a reply, according to the rule by Timo Salmi, does not match the domain in the Message-ID: line (provided there is one).

  3. rejects e-mail entirely in HTML, whether quoted-printable or not, particularly if it contains an unsubscribe request, or if any of the above suspicious booleans are set (it uses procmail scoring technique).
     Note added in Oct 2006 Such rejection was replaced long ago by flagging and dealt with in rc.suspect below. Now mail from legitimate users is processed differently by rc.quote-html.
  4. rejects e-mail with lots of MIME attachments, particulary if one of them is flagged AUDIO, or if it contains a Windows executable (this seems to be a signature of viruses I receive ... they won't harm my Unix system but are annoying). This also uses a scoring technique.
     Awaiting for the updates of our antivirus, this ended up in a catch-all for some known virus or spam signatures. To prevent false positives, TRUSTEDMX is used to bypass rejection for mail coming via trusted IPs.

• Then I use rc.chinese, modified after some rules in the Spam Bouncer, to reject spam in chinese, korean, japanese and turkish. For unclear reasons I was receiving lot of spam in chinese and turkish.

  Note that rejections are sent to different folders (all linked to /dev/null) so that log analysis can keep a statistics of the different reasons for rejection. This applies to the rejections done above or below.

• Then I have a few rules which accept inconditionally incoming mail and may divert it to special folders or programs. They are usually not of interest outside here.

• Rule rc.seminar files away local seminar announcements.
• Rule rc.logs feeds some log messages generated by a crontab into a program disposing of them.
• Rule rc.system is enabled only when I am away, and discards some system messages generated by crontab (which usually I read).
• Rule rc.maillist uses two files mainlists and secondarylists.
  These mailing lists are usually accepted, but when I am away, I divert the main ones to another folder, and I reject altogether the secondary ones (like a temporary unsubscribe). To do this I edit (comment or uncomment) a few lines in .procmailrc

• Then I use rc.conference, with a scoring technique, to accept astronomical conference announcements (announcements tend to have pattern styles resembling spam, like long addressee lists etc., so I must trap them early). I receive them directly when here, or divert them to a separate folder when away.

• In Oct 2006 I added here a new rule rc.quote-html which deals with HTML attachments and improper quotations (the lazy people who quote the entire message they are replying to). Since this can be of general use I provide a separate documentation.

• Then I use rc.suspect, which I won't show, to divert suspicious e-mail. This exploits the pre-analysis done above, and does some more :
  1. a boolean NOTFORME flags e-mail not addressed to my personal address (a candidate for spam according to Timo Salmi)
  2. a boolean TOOMANYTAGS uses a trick by Alan Clifford to flag e-mail with an excess of (possibly fake) HTML tags (which seem a popular way to spam nowadays)
  3. A rule intercepts such (fake or true) HTML as well as MIME encoded HTML and currently rejects such mail (it used to divert it to the "probable spam" folder)
  4. A long rule diverts to a "probable spam" folder e-mail containing some suspicious text patterns (like unsubscribe requests or financial stuff, or medical, porn or scam stuff. It uses a scoring technique, and assign a bias against e-mail not for me, using my old address, with a fake domain (as defined above), or coming from .com .biz .net .org or a commercial .co.xx or .com.xx domain.
  5. A rule intercepts messages generated by antivirus systems and currently rejects such mail (it used to divert it to the "probable spam" folder)
  6. if the above rules are not triggered the relevant scores (MIMESCORE, SPAMSCORE, VIRUSSCORE) are recorded in the mail header
  7. Anyhow e-mail coming from such domains, not for me, or using my old address, is diverted to a "suspect" folder. This simply means I will read it later, and in no case I will send an autoreply (see below) in order not to disclose my new address.
  8. Similarly occurs for "fake address" e-mail, except that the header is labelled differently from the previous case

• Then I use rc.white which uses two files whitelist and gooddomains.
  This unconditionally accepts to my main inbox the e-mail coming from a list of friends (private addresses outside my job arrangements) or from any recognised domain with which we have regular contacts. This includes CNR, observatories and universities in Italy, similar academic institutions (a selection or all of them) in Europe, USA, Japan and Chile, ESA, ESO and NASA.

• I moved to rc.suspect2 the last two steps f (dark green) the processing of suspect mail messages here after the processing of the white list.

• All the rest (i.e potential e-mail coming from individuals in non commercial domains in other countries or institutions) is diverted to a quarantine area requiring an authorization to be accepted and promoted to the white lists.
  This is handled by rc.final (which I won't show so far), which includes the following rules (not necessarily in this order).
  1. the main rule acts on any unidentified mail (with a provision to avoid processing mail-daemon loops as standard according to procmail man pages).
     This rule uses formail cache (file quarantine.cache) to know if the user already contacted us. In this case it generates an unique password (stored in a file Quarantine/.user@domain in a special location and bounces back the message with a standard disclaimer.
  2. in any case this and any future e-mail from the same user will be stored in a dedicated folder Quarantine/user@domain

  3. One other rule deals with authentication requests, i.e. when the user replies with a message containing the password in the subject. The rule checks that the password is associated with the user in the file Quarantine/.user@domain.
     If affirmative the authentication request is validated and sent to the inbox. At the same time also all old mail in the quarantine area are released and sent to the inbox. Finally the user is removed from the cache.

  4. A third rule uses a flag in the subject (added by the previous rule) to let such messages through.

     In fact the user is not yet quarantine-free. At this moment I can read his messages, but I've not inserted him yet in the white lists. Until I do so, any further message from him will be subject to a new quarantine (with a new password).

  ---

• Finally I have some more antispam non e-mail provisions.
  • Never use mailto URLs to compose e-mail from inside a Web browser. And anyhow cloak the "identity" in your browser not to match your valid e-mail address.
  • Do not accept permanent cookies. For this I set my browser cookie file to readonly, so that I do not have to care about accepting cookies manually, but I'm sure they don't survive a browser session.
  • Use some public e-mail address (like despammed.com, spamex.com, or sneakemail.com) in the rare case you have to send something to an unsafe destination (like a survey questionnaire). Then delete it.
  • Never post to Usenet using my own account. I use for this a dedicated account, which is aliased to /dev/null. I may place it in the From: field of the posting, or may use a dummy, not existing address, according to the context. But the X-Sender: field will always be pointing to the valid account aliased to /dev/null.
    Thus I enforce the rule : discussion belongs to Usenet groups, and person-to-person conversation to e-mail, no mixture allowed.