Re: [tlug] state of the art spam filtering

[Curt Sampson <cjs@example.com>]

On 2010-03-16 09:25 +0100 (Tue), Attila Kinali wrote:

> ...on both primary and secondary MX...

Can you define what you mean by "primary" and "secondary" MX? Is there
actually any difference between these servers, besides the priority in
the DNS?

I didn't mention it when I was talking about my configuration, but in
that case it's perfectly reasonable to run all of one's servers at the
same priority.

> My current setup for the high-volume domains is to have strict
> envelope-from/envelope-to checking... and reject everything with a
> 4xx that has an invalid envelope-from, resp 5xx if the envelope-to is
> invalid.

Well, everybody needs to reject things with an invalid envelope-to. What
are you going to do with it if you accept it? :-)

But how do you define an "invalid" envelope-from? As we've seen in
other things that have come up on the list, validity changes from
place to place and time to time. And while there are various checks
you can try to do, none of these guarantee that the address can
actually be delivered. Further, much spam these days does have a
valid envelope-from, it's just some random valid address the spammer
"borrowed" from some poor sod who's going to have to deal with all of
the blowback.

> A nice and cheap filter that also catches quite a lot is the
> requirement to have a valid FQDN in HELO/EHLO (though it does not have
> to resolve).

If it doesn't resolve, how do you know that it's a valid FQDN? By the
RFC standards, , "mail.yahoo.com" is not an *F*QDN because it doesn't
end with a period. ("mail.yahoo.com." would be an FQDN.) But most SMTP
delivery agents don't fully qualify their HELO name with a period.
Conversely, since "blah." is an FQDN (even though it doesn't resolve),
and "com." is (and even does resolve, albeit only to NS records) by that
standard you'd need to accept "HELO blah" and "HELO com".

I have a limited set of local access lists which are used as much for
allowing things as denying them, a handful of header and body checks
that are only there to get rid of the most egregious stuff, and for
the rest I rely on the following SMTP client RBLs, which have done an
excellent job for me:

    sbl-xbl.spamhaus.org
    bl.spamcop.net
    dul.dnsbl.sorbs.net
    web.dnsbl.sorbs.net
    socks.dnsbl.sorbs.net

That still leaves me with a hundred to two hundred spams per day, all
but a few per week of which are caught by spamprobe, which is a Baysean
filter.

cjs
-- 
Curt Sampson         <cjs@example.com>         +81 90 7737 2974
             http://www.starling-software.com
The power of accurate observation is commonly called cynicism
by those who have not got it.    --George Bernard Shaw