Re: [tlug] Email issues

[tlug@example.com]

On Thursday 09 August 2007 19:30:36 Jim Tittsler wrote:
> Which is rather what you would expect given our greylisting implementation.
<SNIP>
> but just scanning through the greylist I see your mail actually arrives from
> more than one IP address, which could impose delays on each.
<SNIP>
> I think greylisting accounts for the delay.

Thank you for the explanation.

> I've not seen that big of a delay...

I wrote a quick script to analyse the delays based on the timestamps in the 
email headers (just because I love mathematics/statistics and it seemed like 
an interesting dataset to look at).  I ran it on my TLUG maildir directory, 
and here are the stats:

# samples: 2559
# errors: 298
mean delay time: 15m
standard deviation: 56m 17s
smallest delay time: 0s
median delay time: 37s
largest delay time: 22h 3m 37s

Note that the errors are emails from Stephen Turnbull, because his SMTP server 
(uwakimon.sk.tsukuba.ac.jp) has a clock skew.  Those emails are ignored for 
the rest of the stats, and # samples does not include errors.

The standard deviation is huge because there are a relatively small number of 
really large values...  Here are the stats with the highest 10% delay times 
trimmed:

# samples: 2304
mean delay time: 4m 18s
standard deviation: 7m 49s
smallest delay time: 0s
median delay time: 27s
largest delay time: 29m 30s

Here is the data for a histogram (of the trimmed version):

     0s ~  2m 57s     1693
 2m 57s ~  5m 54s       15
 5m 54s ~  8m 51s      256
 8m 51s ~ 11m 48s       29
11m 48s ~ 14m 45s       29
14m 45s ~ 17m 42s       32
17m 42s ~ 20m 39s       46
20m 39s ~ 23m 36s       49
23m 36s ~ 26m 33s       70
26m 33s ~ 29m 30s       85

Here is the count for the trimmed samples:
29m 30s ~ 22h 3m 37s   255

The standard deviation is still larger than the mean, but that is to be 
expected given the nature of the problem.

The script used to calculate this data is a quick hack and doesn't even have 
many comments, but I would be happy to share it if anybody is interested.

Cheers,

Travis