L'Xtreme

Are we there yet?

{WD} Postfix content filtering

Table of Contents

Introduction ⇑

This document explains how to filter your incoming email for both spam and virusses using Postfix, Amavisd-new and SpamAssassin. As the Internet gets more common and common to everyone each day, so is the amount of UBE. Massive amounts of spam and virusses are sent to many people (including yours truly) each day. On a “good” day, I receive approximately 50 to 100 spam-messages and between 7 to 30 virusses in my mailbox. Since I do not care about enlarging my willy, or massive amounts of tranquilizers, or a college degree, or willing to open any attachment containing “important documents”, I want to get rid of this junk, now!

Goals of this document ⇑

The purpose of this document is to explain how to implement virus- and UBE-filtering, using Postfix as MTA.

The goals of the used approach are:

  1. Simplicity: we want to keep the impact for the current mail transfer agent(s) as low as possible;
  2. Versatility: since there are many spam and virus scanners, we would like to switch when necessary. In addition, we would like to adapt the behaviour of our content-filter.

Prerequisites ⇑

This document assumes you have got a working Postfix configuration, for example as described by the “Dovecot, Postfix with PAM authentication” how to. The machine used for the described setup is running Debian Linux unstable/testing with the following software installed (Debian packages are enclosed in parenthesises):

In addition, I've written some report scripts in Perl for Amavis. These scripts are mandatory and can be found in the download section.

Content filtering, the basic idea ⇑

Before going into the specific details of implementing content filtering in Postfix, we need to get the basic idea behind the concept of content filtering clear. This will help us to actually understand what we're doing and enables us to debug our setup more easily (see also figure 1):

postfix_content_filter.png

If you replace in the above scenario both MTA-instances with Postfix, and the UBE-filter with a combination of Amavis and ClamAV, you've got the configuration as we tend to implement in this how to.

The central part in our UBE-filter is Amavisd-new. This small daemon, written in Perl, acts as a bridge between the two Postfix instances and scans every incoming message for spam and/or virusses. Reasons for choosing Amavisd-new, is that it integrates nicely with SpamAssassing and has support for various virus-scanners.

Configuring Amavisd-new ⇑

Amavisd-new is configured by a single Perl-module, /etc/amavis/amavisd.conf, which provides a plethora of options. The configuration file provided by Debian has set most of the options to feasible defaults, but some need our attention. The most important ones are:

A snippet with some of the described settings we're using for L'Xtreme.nl:

$mydomain = 'lxtreme.nl';

$forward_method = 'smtp:127.0.0.1:10025';
$notify_method = $forward_method;

# @bypass_spam_checks_acl  = qw( . );

$final_virus_destiny      = D_PASS;
$final_banned_destiny     = D_PASS;
$final_spam_destiny       = D_PASS;

$sa_local_tests_only = 0;
$sa_auto_whitelist = 1;

Note that we pass all found UBE. This is because we're interested in what sort of UBE we're receiving, and allows us to learn the Bayesian filter of SpamAssassin with the results (perhaps after we've checked them).

After you've made all necessary changes to the configuration of Amavisd-new, you need to restart it:

/etc/init.d/amavis restart

Unless you've changed it, you should see a bunch of messages coming from Amavisd-new in /var/log/mail.log. You should check that it is able to use the Mail::SpamAssassin module and has at least one virus scanner found.

Configuring Postfix ⇑

As mentioned, we need to set up a second instance of Postfix. Furthermore, the first instance has to be set up to push everything to our Amavisd-new daemon.

Setting up the connection from Postfix (the first instance) to Amavisd-new and from Amavisd-new to the second instance of Postfix is done in /etc/postfix/master.cf. Append the following snippet:

# Amavisd-new integration
# Make sure e-mails are *not* modified before entering Amavisd...
pre-cleanup unix n      -       -       -       0  cleanup
 -o virtual_alias_maps=
 -o canonical_maps=
 -o sender_canonical_maps=
 -o recipient_canonical_maps=
 -o masquerade_domains=

# 1st instance of MTA => Amavisd-new
smtp-amavis unix -      -       -       -       2  lmtp
 -o smtp_data_done_timeout=1200
 -o lmtp_send_xforward_command=yes
 -o disable_dns_lookups=yes
# 2nd instance of MTA
127.0.0.1:10025 inet n  -       -       -       -  smtpd
 -o content_filter=
 -o local_recipient_maps=
 -o relay_recipient_maps=
 -o smtpd_restriction_classes=
 -o smtpd_client_restrictions=
 -o smtpd_helo_restrictions=
 -o smtpd_sender_restrictions=
 -o smtpd_recipient_restrictions=permit_mynetworks,reject
 -o mynetworks=127.0.0.0/8
 -o strict_rfc821_envelopes=yes
 -o smtpd_error_sleep_time=0
 -o smtpd_soft_error_limit=1001
 -o smtpd_hard_error_limit=1000
 -o receive_override_options=no_header_body_checks

Furthermore, we need to modify both the smtp and pickup services to use our “pre-cleanup” service. We need this service to assure that nothing of the e-mail is modified (like address rewriting, filter, etc.) by any other Postfix service. Modify the existing smtp and pickup services as follows:

smtp      inet  n       -       n       -       -       smtpd
 -o cleanup_service_name=pre-cleanup
pickup    fifo  n       -       n       60      1       pickup
 -o cleanup_service_name=pre-cleanup

The first instance of Postfix needs to push every incoming e-mail to our smtp-amavis service. To do so, we need to add the following snippet to /etc/postfix/main.cf:

# The connection to our UBE filter 
content_filter = smtp-amavis:[127.0.0.1]:10024

Postfix is now completely configured. Restart it to make the settings final:

/etc/init.d/postfix restart

Again, check your log-files (e.g. /var/log/mail.log) to make sure you've made no mistakes and everything is up and running.

Configuring ClamAV ⇑

If you installed ClamAV from one of Debians package repositories, you do not need to configure ClamAV, as it is done automatically for you during the installation.

For security reasons, Debian installs both ClamAV as well as Amavisd-new under their own users and groups (clamav and amavis). In order to make ClamAV to cooperate with Amavisd-new, we need to add the clamav user as a member of the amavis group:

[localhost:~]$ adduser clamav amavis

This allows ClamAV to access the files placed under (potential) quarantine by Amavisd. You can try to do without it, but you will see various messsages in /var/log/mail.log like:

Clam Antivirus-clamd FAILED - unknown status: \
/var/lib/amavis/amavis-20041206T23024 1-08038/parts: Access denied. ERROR

Testing Amavisd-new ⇑

TODO... Eicar pattern, checking log-files & quarantine...

Monitoring and maintainance of Amavisd-new ⇑

TODO... Bayesian learning script, Amavisd-new reporting script...

Downloads ⇑

TODO...

References ⇑

  1. http://kancer.978.org/lawmonkey.org/anti-spam.html

Document history ⇑

Monday, 6 December 2004 (23:32:36) :: © copyright 2004 - L'Xtreme. LXcms v1.0b5.   [valid XHTML 1.1]   [valid CSS]