Re: [tlug] detect fake HTTP referrer

["Stephen J. Turnbull" <stephen@example.com>]

Joe Larabell writes:

 > I can think of two ways to counter-spoof the spoofers.

I don't think any of these work, because NVH doesn't "own" the
legitimate referrers, see below.

 > You could add some pseudo-random string to the URL

Ugh.  This is what cookies are for.  Yeah, I know, some people don't
like to enable cookies.  If so, that pseudo-random string is just
another form of user tracking they won't like, so life sucks all
around.  Better to tell them up front that they need to enable cookies
for your site.

 > The other way would be to record the IP address of the visitor when he 
 > accesses your d/load page

The point of NVH's story is that for legit referrers, the visitor just
downloads the file.  It's like an OEM agreement, I suspect: he lets
"people he likes" rebrand his content (ie, by linking directly to it
from their pages).

A more complex scheme that might work (haven't thought about it
carefully) is to have the authorized referrer actually be a
transparent proxy (any Apache 2 site can do this).  Then you simply
refuse accesses to the direct public URL from anywhere that is not on
your authorized referrer list.  In order to avoid stealing bandwidth
from your referrers, the link on your server that gets sent back via
proxy is a temporary redirect to a direct URL to your server, probably
one synthesized for this transaction (or you could use cookies, I
suppose).

 > BTW, the more common form of this "theft" is when pages link directly to 
 > image files stored on some other machine -- either out of laziness or a 
 > desire to keep their own bandwidth to a minimum by serving their images 
 > from someone else's site. I believe the correct term is "bandwidth theft".

Actually, in the U.S. the correct term "obeying copyright law without
checking the license".  If somebody puts up content on a public site
with no access controls, then anybody may download it.  This *does
not* mean that "anybody" may keep anything more than the "ephemeral"
copies that are required to view it.  Let alone redistribute.

This is identical to the policy of most mailing lists that you do not
copy a full text if there's an URL available.  Sure, it saves
bandwidth, but the overriding reason (and the reason "inclusion-by-
reference" cannot be prohibited by law or custom, but only controlled
on a case-by-case basis) is to avoid the (high!) cost of verifying
license policy.

 > I've had my share of bandwidth thieves. I implemented a CGI to prevent 
 > this on my site for images but... it too depends on the Referer string, 
 > which, as you now know, is both unreliable and easily spoofed.

On every page that contains images, set a cookie with a short expiry
(say 1 hour), and insist on the cookie before you give away an image.