Re: [tlug] Do you whitelist or blacklist utf-8?

[Shmuel Fomberg <owner@example.com>]

Hi Dave.


Assuming that you want to avoid SQL-Injection attacks,


move the input to the target encoding before doing any filtering.

doing filtering in different encoding is malpractice.


after it is in the target encoding, you probably want to examine only

characters that are in the ascii range.

if your encoding is utf-8, you can write a tight loop that examine the MSB

of a byte, and pass this byte if it is set. else - whitelist / blacklist 
this byte.


IMHO, only whitelist.


Of course, all this is not excuse for not using pre-compiled SQL queries

with placeholders, or whatever they are called in PHP.


Good luck,

Shmuel.


On 2011/02/22 12:57, Dave M G wrote:

> TLUG,
>
> I've been going a little mental today trying to figure out how to filter
> out possible malicious characters from POST data going to my site. I
> want to block things like<,>, *. etc...
>
> The thing is that I also want to be able to allow CJK characters, and
> any other language with non-Latin characters. This is a snap to do if
> you just want to allow 0-9a-zA-Z. But once you get into Unicode land, it
> seems to be a whole other ballgame.
>
> I've got three stages I want to filter on. First I want to block
> characters on the client side with Javascript, so that the user is aware
> of what characters are permissible when entering names and whatnot. Then
> I want to block any bad characters on the server side in PHP to make
> sure no script kiddies have tried to POST anything nasty. And also, just
> for good measure, I want to ensure no nastiness is inserted into my MySQL.
>
> I'd like all three steps to be consistent with each other, so I'm trying
> to standardize a set of bad characters that I can filter for at each step.
>
> However, where I've broken down is whether or not I should blacklist bad
> characters (where I fear I might miss one), whitelist good characters
> (seems tough to get a whitelist that's utf-8 compatible), or do
> something like make comparisons on HTML entities or with regex or
> something using built in functions (PHP and Javascript differ on
> specific functions and their results).
>
> Since you guys are the go-to people for handling utf-8 text, I thought
> maybe you've encountered this before.
>
> How do you handle filtering malicious code from utf-8 text that contains
> CJK and other languages?
>
> And how do you do it in PHP and Javascript?