[tlug] Strange TCP problem, SACK?

[Alan Busby <thebusby@example.com>]

A network problem, likely quite simple. is killing me slowly and painfully here. 

Until recently I've connected to a remote server through a vpn tunnel (branch office to home office) without any problems. Suddenly this week, after a power failure, I'm getting strange TCP disconnect errors between multiple different servers/protocols/OSes. Based on the fact that the only consistent point seems to be the network, I've focused my efforts there and was able to record a failure on both the client and server via tcpdump.

My network skills are pretty rusty though, and I hoping someone here can help me identify what's going on here.
Could a firewall router be interfering and causing this?


From the client (Linux ubuntu 2.6.22-14-generic):
18:58:55.803519 IP client.50624 > server.ssh: P 2545:2593(48) ack 2313 win 93 <nop,nop,timestamp 49023574 110651213>
18:58:55.926181 IP server.ssh > client.50624: . ack 2593 win 128 <nop,nop,timestamp 110651245 49023574>
18:58:55.937920 IP server.ssh > client.50624: P 2313:2393(80) ack 2593 win 128 <nop,nop,timestamp 110651248 49023574>
18:58:55.938207 IP client.50624 > server.ssh: P 2593:2625(32) ack 2393 win 93 <nop,nop,timestamp 49023587 110651248>
18:58:55.951162 IP server.ssh > client.50624: P 2393:2441(48) ack 2593 win 128 <nop,nop,timestamp 110651251 49023574>
18:58:55.951414 IP server.ssh > client.50624: P 2441:2505(64) ack 2593 win 128 <nop,nop,timestamp 110651251 49023574>
18:58:55.951435 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023589 110651251>
18:58:56.072085 IP server.ssh > client.50624: P 3865:3929(64) ack 2625 win 128 <nop,nop,timestamp 110651281 49023589>
18:58:56.072096 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023601 110651251,nop,nop,sack 1 {3865:3929}>
18:58:56.085826 IP server.ssh > client.50624: P 3929:3993(64) ack 2625 win 128 <nop,nop,timestamp 110651285 49023589>
18:58:56.085838 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023602 110651251,nop,nop,sack 1 {3865:3993}>
18:58:56.093570 IP server.ssh > client.50624: P 3993:4057(64) ack 2625 win 128 <nop,nop,timestamp 110651287 49023589>
18:58:56.093579 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023603 110651251,nop,nop,sack 1 {3865:4057}>
18:58:56.193257 IP server.ssh > client.50624: P 4057:5145(1088) ack 2625 win 128 <nop,nop,timestamp 110651311 49023601>
18:58:56.193265 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023613 110651251,nop,nop,sack 1 {3865:5145}>
18:58:56.312181 IP server.ssh > client.50624: P 5145:5209(64) ack 2625 win 128 <nop,nop,timestamp 110651341 49023613>
18:58:56.312191 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023625 110651251,nop,nop,sack 1 {3865:5209}>


>From the server (Linux ubuntu 2.6.22-14-server):
11:14:10.125386 IP client.50624 > server.ssh: P 2545:2593(48) ack 2313 win 93 <nop,nop,timestamp 49023574 110651213>
11:14:10.125446 IP server.ssh > client.50624: . ack 2593 win 128 <nop,nop,timestamp 110651245 49023574>
11:14:10.137229 IP server.ssh > client.50624: P 2313:2393(80) ack 2593 win 128 <nop,nop,timestamp 110651248 49023574>
11:14:10.150360 IP server.ssh > client.50624: P 2393:2441(48) ack 2593 win 128 <nop,nop,timestamp 110651251 49023574>
11:14:10.150558 IP server.ssh > client.50624: P 2441:2505(64) ack 2593 win 128 <nop,nop,timestamp 110651251 49023574>
11:14:10.257557 IP client.50624 > server.ssh: P 2593:2625(32) ack 2393 win 93 <nop,nop,timestamp 49023587 110651248>
11:14:10.257599 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110651278 49023587>
11:14:10.269806 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023589 110651251>
11:14:10.271178 IP server.ssh > client.50624: P 3865:3929(64) ack 2625 win 128 <nop,nop,timestamp 110651281 49023589>
11:14:10.285259 IP server.ssh > client.50624: P 3929:3993(64) ack 2625 win 128 <nop,nop,timestamp 110651285 49023589>
11:14:10.292828 IP server.ssh > client.50624: P 3993:4057(64) ack 2625 win 128 <nop,nop,timestamp 110651287 49023589>
11:14:10.390779 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023601 110651251,nop,nop,sack 1 {3865:3929}>
11:14:10.390814 IP server.ssh > client.50624: P 4057:5145(1088) ack 2625 win 128 <nop,nop,timestamp 110651311 49023601>
11:14:10.404700 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023602 110651251,nop,nop,sack 1 {3865:3993}>
11:14:10.411536 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023603 110651251,nop,nop,sack 1 {3865:4057}>
11:14:10.411560 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110651316 49023603>
11:14:10.511323 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023613 110651251,nop,nop,sack 1 {3865:5145}>
11:14:10.511357 IP server.ssh > client.50624: P 5145:5209(64) ack 2625 win 128 <nop,nop,timestamp 110651341 49023613>
11:14:10.630372 IP client.50624 > server.ssh: . ack 2505 win 93 <nop,nop,timestamp 49023625 110651251,nop,nop,sack 1 {3865:5209}>
11:14:10.747154 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110651400 49023625>
11:14:11.419178 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110651568 49023625>
11:14:12.763226 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110651904 49023625>
11:14:15.451350 IP server.ssh > client.50624: P 2505:3865(1360) ack 2625 win 128 <nop,nop,timestamp 110652576 49023625>
...


Thank you,
    Alan