Re: [tlug] Tokyo ISP Throttling? Why does my internet speed increase so much by setting up a proxy?

[Tim Osburn <tim@example.com>]

On Sun, 21 Feb 2016, Jawaad Mahmood wrote:

> I'm very confused about this.
> I am running NTT Flez Hikari to my mansion; it's slow as molasses.
>
> However, today I setup a Squid3 proxy server on a cloud server I own. When I 
> connected to that (and connected my wife), all of a sudden all sorts of slow / 
> dropped connection problems went away.

Jawaad,
	Sorry for the long reply, this Email sorta grew and got out of hand. :)

	You mentioned NTT Flez Hikari, which I believe NTT only provides the 
line/pipe between your home modem and the provider/ISP (NTT is basically like a 
layer 2 provider in a sense). Then you also have the provider/ISP which you 
didn't mention which provides the internet over that pipe (layer 3 and up). If 
it's not consistent I would probably rule out NTT as the issue and look at your 
ISP. But that may not always be the case.

	Who is your ISP
	Is your traffic to your squid proxy encrypted?

1) Are you using WIFI at home when there is slowness? There could be RF
   interference from neighbors Access Points or cordless phones if that's the
   case causing poor performance and packet loss. If so try plugging in a cable
   and not using wifi.

2) Perhaps your route to the destination has changed. It's possible your
   provider/ISP has poor internet peering to their providers and the route to
   where you are going is sub optimal or the peering point is congested,
   dropping packets or taking on errors. You might also provide a 100 iterations
   for the destinations through a program called MTR (Matt's TraceRoute). It
   might help show you points along the path that may be congested or suspect of
   poor quality.

	Unix
	https://www.bitwizard.nl/mtr/
	https://github.com/traviscross/mtr

	Windows
	http://winmtr.net/download-winmtr/
	https://sourceforge.net/projects/winmtr/files/Latest/

3) Your provider/ISP may be traffic shaping, throttling or doing some sort of
   bandwidth management/deprioritization on specific traffic that is effecting
   your through put when not using your proxy. Then when you use your proxy they
   don't see that traffic or touch it since it's either on a unknown WEB port or
   because it's encrypted and they have no idea what it is. A large mobile
   provider in the US did this or may be still doing this on mobile phones. They
   also at one pointed added in additional identifiable headers to web requests
   that were not encrypted when you browsed to sites.  Something like the below
   can allow you to detect that.

	Example:
		http://www.osburn.com/info/
		https://www.osburn.com/info/

	Is it also possible your proxy is compressing your data which in turn 
results if typically what looks like faster connectivity?

4) It's possible they are doing packet inspection so when you tunnel to your
   device using TLS/SSL they can not see into that so they just pass it without
   inspecting or adding latency to that.

5) Have you eliminated the possibility of something "local" between your PC and
   your NTT Modem that might be effecting your traffic? In line or software
   based? Try rebooting your router and/or modem. Disable your firewall/anti
   virus software as quick test. Does it only effect your computer? Also it's
   very important to also check your MTU settings on your device that connects
   to your provider if that they are doing any sort of encapsulation between you
   and them. Not having that right will result in dropped packets and what might
   seem as slow connectivity. Your provider should be able to tell you what your
   MTU should be, if PPPoE then you might try a MTU setting of 1454 instead of
   the tyical default ~1490.

6) Like others have mentioned it is also possible others in your building could
   be consuming lots of internet as well. If your mansion is anything like ours
   where there are lots of people in it, it's likely NTT has several fibers into
   your building. On each fiber (for example our building) they typically will
   put up to 16 customers on a single 1GigE fiber. Customers in our building
   have several speeds to choose from which max out at 100mbps up/down over. It
   could be if you complain and they have multiple fibers into the building,
   they might be able to move you over to another fiber that might not be as
   congested as the one you are on. Of course that's only if that's really is
   the case. It also may take several calls and complaints to motivate them to
   do that also as they will need to send someone out to your building to
   physically move your connection over to another node. Also in our building
   NTT is not the only "line" fiber/teleco. At our building there 2 other
   telecos that have fiber into our building. Then also each of those telecos
   have a list of providers/ISPs you can then choose from. Lots of choices and
   sometimes hard to know which to pick. So maybe you might have more options
   then you know, talking to your building manager might help you discover if
   you have additional choices other then NTT as a line telco. Switching
   providers may also be what you need to do if it turns out to be the provider
   and not the line carrier.

Just to clarify the terms:
LINE = The circuit between your outlet on the wall to your ISP Provider
ISP/PROVIDER = the ISP that is proving you IP access to the internet (layer 3)

	I've been told that in Japan, these are typically not the same 
companies. Where as in the US they typically are the same company.

6) You just might give your provider/ISP a call and tell them whats happening,
   they may have something miss-configured or know about a sub optimal routing
   issue that is effecting their customers.

7) Do you have a encrypted VPN service you can also test, how does it compare
   when using that? Try MTR with and without a full on encrypted VPN and see how
   your times look using that as that will force you to take a different path.

Tim Osburn
http://www.m2os.com
W7RSZ / JG1MBR

https://instagram.com/tim.osburn/