Re: [tlug] detect fake HTTP referrer

[Shin MICHIMUKO <smitimko@example.com>]

Hi,

I believe the Referer header doesn't exist if the link is written in the
e-mail.

Nguyen Vu Hung wrote:
> Yes, some hosting sites block "hot linking", or "hot download", that
> means, you have to visit that site and click a link to download. We
> can also do this with mod_rewrite ,while this can be bypassed if you
> set Referer [1] which is easily done with Perl, PHP or wget.

I guess that you may want to block by Referer or by IP addresses.
Otherwise you may need to use Cookie or a query string to authorize the
requests.

>>> so that the request looks "normal". They are stealing my traffic.
>> I'm not an expert in HTTP, but i doubt that this is possible.
>> Referers are a client side thing and just some info for the
>> server where they are comming from.
> No.
> 
> For example, aocgroup.com.ar[2] with my permission, create a list that
> includes links to *all* for files under /aoc/recs. If you don't set
> Referer, we will not able to know where the traffic comes from. That's
> why we call it "traffic stealing".

If the link is introduced in a mailing list, then the request may not
have the Referer. So it can be happened, I think. Also, some of the
mirroring tools can start pulling the contents from the top directory
without Referer, so I think it can be happened.

> I can't do that - in fact it is impossible -  because my site serves world-wide.

Sometimes the IP address of the clients in China are behind the
firewall. Even if the IP address blocks are very limited range, the
client could be so many. Sometimes they are using the proxy servers
(sometimes forced to use it by the political reasons), and it is
sometimes difficult to determine if those requests are really invalid or
not.

If you think that the requests are really invalid, then I recommend to
block the several IP addresses for the moment. If the end users will
have the actual problem, then they may contact you to allow the accesses.

If you think that the requests are valid and if you don't want to block
them, then you may need to add more capacity to the server farm (like a
CDN).

How do you think?

Regards,
---Shin.
-- 
== Money is one of the minimum conditions to do anything, but... =======
  Shin MICHIMUKO <smitimko@example.com> http://www.peanuts.gr.jp/
============================================ Freedom is everything. ====