Re: [tlug] "UTF-8 & ISO-2022-JP"

[Andrew Hamilton <awh@example.com>]

On Mon, Dec 05, 2005 at 02:01:22PM +0900, Lyle (Hiroshi) Saxon wrote:
> my e-mail client (Mozilla) set now is to UTF-8 for default messages and 
> then I manually select ISO-2022-JP when I have a J-text message to send 
> someone who is using software/settings/provider/whatever that only seems 
> to be happy with ISO-2022-JP.

ISO-2022-JP has pretty much been the "network encoding" for email messages
(and IRC, for that matter) for as long as I've been worrying about Japanese
character encodings (10 years or so).  Any of the "native japanese" 
mailers -- Becky, as well as Outlook and Outlook Express (when the system 
locale is set to Japanese), as well as Japanese ports of unix mailers (mutt,
pine, etc.) all convert from the system encoding to ISO-2022-JP when 
sending mail, as well as setting the "Content-Type:  ~~~~~~~~; 
charset="ISO-2022-JP" header in the mail.

Basically, you get mojibake when sending non-ISO-2022-JP mail to people,
because Japanese mail software expects Japanese mail to be encoded with
ISO-2022-JP.  Why?  Because that's the way it's always been.

If you think about the differences between HTTP and Email/IRC you'll see
that it makes sense..  With HTTP you have a direct connection between the
client and server; and the protocol makes it very easy to send Unicode.

With Email and IRC you have messages relayed from the source via many servers
to the destination.  Those many servers all may be running different software,
on different OSes, with different versions, some of which may well have been
set up before the need for 8-bit-safe communications was really well-known.
Nothing about either protocol (IRC or SMTP) says that any of the communication 
has to be 8-bit-safe, so it's certainly possible that someone is running a
fully standards-compliant server that still fails to transmit EUC, SJIS, or
UTF-8.  ISO-2022-JP is 7-bit only, and the most "weird" character is escape, 
which pretty much anything can deal with.  

 - awh