Re: [tlug] detect fake HTTP referrer

[Joe Larabell <fred62@example.com>]

> > I'm not an expert in HTTP, but i doubt that this is possible.
> > Referers are a client side thing and just some info for the
> > server where they are comming from.
> No.
>
> For example, aocgroup.com.ar[2] with my permission, create a list that
> includes links to *all* for files under /aoc/recs. If you don't set
> Referer, we will not able to know where the traffic comes from. That's
> why we call it "traffic stealing".

Yeah... the Referer is controled from the client side. There is little you 
can do to prevent someone from spoofing the Referer string. Moreover, the 
browser is not even required to send Referer (and possibly some obscure 
web clients do not).

I can think of two ways to counter-spoof the spoofers. You could add some 
pseudo-random string to the URL (and also to the path where the file is 
located). Once in a while (like midnight, with a cron job) rename the 
directory where the file is located and re-generate the download page with 
the new string in the download URL. The spoofers will be able to spoof for 
a while but they will have to re-publish the fake Referer every 24 hours, 
which will probably be too much trouble.

The other way would be to record the IP address of the visitor when he 
accesses your d/load page (using either session management or a simple CGI 
that just writes the IP address and access time to a file). Then, put the 
d/load file behind a CGI and before serving up the file, check to make 
sure the IP address is one from which your download page was recently 
viewed. A variant on this would be to ask the visitor to login (which, 
most likely, either stores the IP or registers a cookie) before they can 
download the file.

Speaking of cookies, another possibility would be to set a cookie but if 
the value is fixed (ie: neither random nor time-dependant), that would 
also be pretty easy to spoof after a couple of experimental downloads to 
discover the string.

BTW, the more common form of this "theft" is when pages link directly to 
image files stored on some other machine -- either out of laziness or a 
desire to keep their own bandwidth to a minimum by serving their images 
from someone else's site. I believe the correct term is "bandwidth theft".

I've had my share of bandwidth thieves. I implemented a CGI to prevent 
this on my site for images but... it too depends on the Referer string, 
which, as you now know, is both unreliable and easily spoofed.

---
Joseph L (Joe) Larabell            Never fight with a dragon
http://larabell.org                     for thou art crunchy
                                  and goest well with cheese.