[tlug] Re: Hotmail mail encoding

["Jim Breen" <jimbreen@example.com>]

"Stephen J. Turnbull" <stephen@example.com> dashed off:
> Are you sure that's not
>
> <DIV =
> class=3DRTE>=E2=80=9A=C2=B1=E2=80=9A=C3=B1=E2=80=9A=C3=89=E2=80=9A=C2=BF=E2=
> =80=9A=C3=8D=C2=81B</DIV>
>
> (note how the leading "=" of the second line becomes a trailing one in
> the first line; line breaks are significant here)?
>
> If it's the former, it's a very broken quoted-printable encoding (ie,
> ASCII armor) that any self-respecting MUA will refuse to decode.  If
> the latter, it's compatible with UTF-8 encoded with quoted-printable,
> decoding to gibberish (as far as I can tell).

'T'ain't Japanese in UTF-8. UTF8ed Japanese is always in 3-byte sequences
starting with En (apart from some of the rare kanji that turned up in
JIS213). There are 28 bytes in that sequence, and the last is a plain old
"B" which can't be part of UTF8. Smells like Shit-JIS.

> That is correct given the headers you post for that message.  That is
> perfectly valid text/plain as is; no decoding is requested, and all
> characters are ASCII so no error will be signaled.  Of course, it's
> gibberish to humans, but that is not your MUA's fault, it is being
> *lied* to.  What appears to be happening is that Hotmail is assuming
> that the remote end will treat the message as HTML no matter what the
> headers say.

Exactly.

> I suspect that you can read the message in Gmail because Gmail embeds
> it in an HTML element before sending it (but without escaping the
> entities, which is arguably broken).  (Or maybe your browser
> implicitly does so.)  Evolution doesn't read it because it's an
> approximation to a conforming MUA, but has no inherent bias toward
> displaying things as HTML to cover up for Hotmail's total bogosity.

Gmail is pretty good at conforming to mail standards. I sent myself
an email in Japanese from Hotmail to Gmail. It arrived quite
viewable, but I couldn't inspect the encapsulation. I then forwarded it to
a site with a text client and it arrived in ISO-2022-JP (I have Gmail
set to default for the language, which means the presence of any kana
flips it to ISO-2022-JP.)

>  > Could the fact that I set my default encoding in Firefox to UTF-8 be
>  > related to this?
>
> Yes, but "how" is a good question.  Hotmail is clearly incredibly
> broken, but webmail is such a complex application it's hard to guess
> exactly where the breakage is occurring.

I think the Firefox setting is irrelevant when using Hotmail.

Cheers

Jim
--
Jim Breen
Honorary Senior Research Fellow
Clayton School of Information Technology,
Monash University, VIC 3800, Australia
http://www.csse.monash.edu.au/~jwb/