Re: [tlug] "UTF-8 & ISO-2022-JP"

["Lyle (Hiroshi) Saxon" <ronfaxon@example.com>]

Andrew Hamilton wrote:

> On Tue, Dec 06, 2005 at 02:46:31PM +0900, Lyle (Hiroshi) Saxon wrote:
>  
>
> > Another bit of text from the (external) general discussion (I won't make 
> > a habit of doing this, but think it's relevant in this case). - Lyle
> >    
>
> I would hope that if you decided to post anything that I wrote anywhere
> else, you would be thoughtful enough to attribute it.
>  

You're right.  I shouldn't have.  It's the only thing I've ever posted 
anywhere from this list, but  - yes - I did post the following - and I'm 
sorry, I won't do it again.  I posted it without any identifying 
information.... if you'd like me to put your name on it, I can go back 
and do that.....

Lyle


ISO-2022-JP has pretty much been the "network encoding" for email messages
(and IRC, for that matter) for as long as I've been worrying about Japanese
character encodings (10 years or so).  Any of the "native japanese"
mailers -- Becky, as well as Outlook and Outlook Express (when the system
locale is set to Japanese), as well as Japanese ports of unix mailers (mutt,
pine, etc.) all convert from the system encoding to ISO-2022-JP when
sending mail, as well as setting the "Content-Type:  ~~~~~~~~;
charset="ISO-2022-JP" header in the mail.

Basically, you get mojibake when sending non-ISO-2022-JP mail to people,
because Japanese mail software expects Japanese mail to be encoded with
ISO-2022-JP.  Why?  Because that's the way it's always been.

If you think about the differences between HTTP and Email/IRC you'll see
that it makes sense..  With HTTP you have a direct connection between the
client and server; and the protocol makes it very easy to send Unicode.

With Email and IRC you have messages relayed from the source via many 
servers
to the destination.  Those many servers all may be running different 
software,
on different OSes, with different versions, some of which may well have been
set up before the need for 8-bit-safe communications was really well-known.
Nothing about either protocol (IRC or SMTP) says that any of the 
communication
has to be 8-bit-safe, so it's certainly possible that someone is running a
fully standards-compliant server that still fails to transmit EUC, SJIS, or
UTF-8.  ISO-2022-JP is 7-bit only, and the most "weird" character is 
escape,
which pretty much anything can deal with.