Re: [tlug] state of the art filtering

[Tobias Diedrich <ranma@example.com>]

Tobias Diedrich wrote:
> Would be interesting to check the logs and see how big the
> percentage of mail servers that didn't come back after being denied
> by greylisting really is...

Using a slightyly modified
http://ftp.linux.cz/pub/linux/people/jan_kasprzak/postgreystats.pl

|Greylisting stats from Mar 15 07:06:18
|                  till Mar 16 07:21:24
|
|Messages
|total:                       240
|accepted immediately:        104 ( 43.3%)
|greylisted:                  136 ( 56.7%)
|
|SMTP servers:
|                                   hosts   messages
|No message accepted:                  60        113
|No message greylisted:                68         90
|All accepted (maybe delayed):          6          1 = 8-7
|More blocked than accepted:            6         10 = 16-6
|
|Greylisting performance:
|
|Greylisted:            136
|Never accepted:        113 ( 83.1%)

So it looks like postgrey reduced the number of spam messages by 83%
in this case.

#!/usr/bin/perl -w
#
# Postgreystats - a simple and incorrect statistics package for Postgrey.
# Use at your own risk.
# Distributable under the terms of GNU General Public License, version 2.
# Written by Jan "Yenya" Kasprzak, http://www.fi.muni.cz/~kas/, 2006-11-10
#
# Usage: postgreystats.pl /var/log/maillog
#
my %clients;

my ($delay_total, $delay_msgs);
my ($msgs_gray, $msgs_in);

my ($fromtime, $totime);

my $prev;

$delay_msgs = 0;
$delay_total = 0;

while(<>) {
	$prev = $_;
	if (!defined $fromtime) {
		($fromtime) = m{^(... .. ..:..:..)};
	}
	if (my ($client) = m{postfix/smtpd\S+: [0-9A-F]{10,12}: client=\S+\[(\d+\.\d+\.\d+\.\d+)\]}) {
		$clients{$client}->{in}++;

		$msgs_in++;
		next;
	}

	if (my ($client) = m{postfix/smtpd\S+: NOQUEUE: reject: RCPT from \S+\[(\d+\.\d+\.\d+\.\d+)\]:.* 450 .*Greylisted}i) {
		$clients{$client}->{grey}++;
		$msgs_gray++;
	}

	if (my ($delay) = m{postgrey\S+: delayed (\d+) seconds: }) {
		$delay_total += $delay;
		$delay_msgs++;
	}
}

($totime) = ($prev =~ m{^(... .. ..:..:..)});

print "\nGreylisting stats from $fromtime\n",
	"                  till $totime\n\n";

$msgs_in   -= $delay_msgs;
$msgs_gray -= $delay_msgs;
my $msgs_total = $msgs_gray + $msgs_in + $delay_msgs;

printf   "Messages\n"
	."total:                %10d\n"
	."accepted immediately: %10d (%5.1f%%)\n"
	."greylisted:           %10d (%5.1f%%)\n\n",
	$msgs_total,
	$msgs_in,    100*$msgs_in    / $msgs_total,
	$msgs_gray,  100*$msgs_gray  / $msgs_total;

#printf   "Greylisting delay (avg): %10ds delayed\n"
#	."                         %10ds all\n\n",
#	int($delay_total/$delay_msgs),
#	int($delay_total/$msgs_in);

my ($zero_in, $zero_grey, $grey_pos, $grey_neg);
my ($zero_in_msgs, $zero_grey_msgs, $grey_pos_msgs_in, $grey_pos_msgs_gr,
	$grey_neg_msgs_in, $grey_neg_msgs_gr);

for my $client (keys %clients) {
	my $in = $clients{$client}->{in}   || 0;
	my $gr = $clients{$client}->{grey} || 0;

	if ($in == 0) {
		$zero_in++;
		$zero_in_msgs+=$gr;
	} elsif ($gr == 0) {
		$zero_gr++;
		$zero_gr_msgs += $in;
	} elsif ($gr <= $in) {
		$grey_pos++;
		$grey_pos_msgs_in += $in;
		$grey_pos_msgs_gr += $gr;
	} else {
		$grey_neg++;
		$grey_neg_msgs_in += $in;
		$grey_neg_msgs_gr += $gr;
	}
}

my ($grey_pos_msgs, $grey_neg_msgs) = (
	$grey_pos_msgs_in - $grey_pos_msgs_gr,
	$grey_neg_msgs_gr - $grey_neg_msgs_in
);

printf   "SMTP servers:\n"
	."                                   hosts   messages\n"
	."No message accepted:          %10d %10d\n"
	."No message graylisted:        %10d %10d\n"
	."All accepted (maybe delayed): %10d %10d = %d-%d\n"
	."More blocked than accepted:   %10d %10d = %d-%d\n\n",

	$zero_in, $zero_in_msgs,
	$zero_gr, $zero_gr_msgs,
	$grey_pos, $grey_pos_msgs_in-$grey_pos_msgs_gr, $grey_pos_msgs_in,
		$grey_pos_msgs_gr,
	$grey_neg, $grey_neg_msgs_gr-$grey_neg_msgs_in, $grey_neg_msgs_gr,
		$grey_neg_msgs_in;

printf  "Graylisting performance:\n"
	."\n"
	."Graylisted:     %10d\n"
	."Never accepted: %10d (%5.1f%%)\n",

	$msgs_gray,
	$zero_in_msgs,  100*$zero_in_msgs/$msgs_gray;

-- 
Tobias						PGP: http://8ef7ddba.uguu.de