Re: tlug: XEmacs and Kanji detection

[Steve Dunham <dunham@example.com>]

--------------------------------------------------------
tlug note from Steve Dunham <dunham@example.com>
--------------------------------------------------------
"Stephen J. Turnbull" <turnbull@example.com> writes:

> I've never found locale to be useful for much of anything; kterm
> hardcodes it, and GNU Mule ignores it.  So this didn't occur to me.
> (Who looks lame now?)  Yeah, that seem to do the trick (dammit; it
> means that getting m17n right is going to be really mendoukusai;
> locale-related functions are dispersed all over xemacs, but not in
> Mule source files :-( ), although I'm using an XIM-based XEmacs, not a
> standalone henkan server version.

You can also do thus by selecting "Options/Language Environment/Japanese".
I believe the issue is that some of the Japanese hooks don't get
loaded until this is done.

I believe the locale is passed to the input method. (I suspect this is 
why the Solaris input method didn't work for me.)

> The locale processing also seems to be inconsistently implemented,
> since "LANG=ja_JP.EUC" results in SJIS being correctly displayed (and
> "LANG=ja_JP.SJIS" produces correct results for EUC).  

Umm, what do you want it to do? Disable EUC support?

Currently it only uses the LANG variable to determine the "default"
language for startup.  Hence, it only uses the language part of the
variable (the "ja").  (Somebody has beta support localized menus &c,
but I don't know how to enable it.)

XEmacs, by default, seems to have iso-2202 filters in the loading
process.  The SJIS detection seems to be added when you load the
japanese language stuff.  This happens when LANG=ja or you pick the
Japanese menu item. (You can probably do this in .emacs too, you
should be able to select Japanese and "Save Options", but I haven't
tried it.)

Anyways, you can change the "default" language in the menu:
"Options/Language Environment".

File encoding can be set on a per buffer basis using C-x C-n f (type
C-x C-n C-h for a list of bindings).

> Furthermore, using `(setenv "LANG" "C")' or any other locale does
> not affect this, so the locale of the XEmacs process seems to be
> fixed at invocation, and only used to invoke Mule features.

Ahh, digging around, I found yet another reason for this: in the lisp
directory, there is a locale/ja/locale-start.el.  Apparently, there is 
no directory for other locales.  (XEmacs desperately needs testers for
non-japanese MULE stuff.)

> (I suspected that XEmacs would handle ISO-2022-JP when compiled
> without Mule, because it gets it right when EUC and SJIS are
> mojibake.  But that is not the case.)  It's really not clear to me
> what all is going on here.  The Mule features of XEmacs seem to be
> really hacked in.

Why do you get a feeling that the mule features are hacked in?  It
feels fairly clean to me.  There are probably some necessary
differences from the gnu emacs MULE, because of design differences in
the core editor.  But as far as I can tell, MULE is nicely integrated
into the editor.

> Re: w3.el.  I'm _not_ getting Japanese on the Mule homepage (the top
> page is OK, since all the Japanese is in GIFs, but
> http://www.etl.go.jp/Research/mulepage/MuleJapanese.html bombs).  I
> don't seem to have any problems with the W3 multilingual example page
> (http://www.ntt.co.jp/japan/note-on-JP/multi-example.html) except for
> missing fonts.

> It seems to choke on the (standard-compliant) MIME content-type
> header.  If that (`Content-type: text/html; charset=iso-2022-jp') is
> present, the non-ASCII characters turn into mojibake :-(.  (I've
> confirmed this by prepending that to the header of multi-example.html
> and getting mojibake, and taking out the charset reference in the
> Content-type of MuleJapanese.html and getting Japanese.)  Sigh.

Sounds like a bug in w3.el... It seems to be ignoring the charset
specification in the HTTP header.  Does it work in MULE?

You can dig around an figure out why, if you want.  (It would be
rather low on my list of things to do - I use netscape for WWW.  The
top of my todo list for XEmacs would be printing...)

> w3.el doesn't do JPEGs.  Pity, that.

Is your XEmacs linked with libjpeg?  I'm looking at my (rather lame)
homepage which includes a jpeg, and it looks fine.   It even handles
my style sheet.
  http://dunham.tcimet.net/~dunham/

> Craig, I have a couple of questions.

> Do you get the right charsets when switching from an SJIS server to an
> ISO-2022 server in w3.el?  Do you need to do anything else (in
> particular I'm thinking of the liblocale dodge that works with
> Netscrap)?  Do you get messages about not being able to set locale,
> using C/POSIX instead?  (I do, unless I use LD_PRELOAD=liblocale.so,
> since I have XIM compiled in, and any keyboard input gives instant
> crash.  At least it's for reasons I already understand.  I think; if I 
> do understand, you probably don't get those messages or need to use
> liblocale.so with XEmacs/Mule/Canna.)

This is the funny thing: I get those messages from XEmacs for
"LANG=de", perl does the same thing.  But I don't get any messages
from Netscape (which displays correctly localized text) or various GNU 
utilities (again with varying degrees of localized text).

I fully expect that message for LANG=ja, since I don't have any
"ja" locale in /usr/share/locale.  (This needs to be fixed.) This is
why the liblocale.so is needed. Apparently, it calls some internal X
functions to trick it into thinking it has a wide character locale on
systems lacking the "ja" locale (English Solaris ships without it).

I believe the X locale functions will work in conjuction with libc
locale, if you have a bona fide "ja" locale - I'll try to look into
why libc doesn't have a "ja" locale and why the latin-1 locales don't
work in some instances.

> Lists of all defined locales and aliases are in
> /usr/X11R6/lib/X11/locale/locale.{dir,alias}.

The X locale information is there.  The libc locales are in
/usr/share/locale (on Debian).


スチェヴ・ダーナム
dunham@example.com

-----------------------------------------------------------------
a word from the sponsor will appear below
-----------------------------------------------------------------
The TLUG mailing list is proudly sponsored by TWICS - Japan's First
Public-Access Internet System.  Now offering 20,000 yen/year flat
rate Internet access with no time charges.  Full line of corporate
Internet and intranet products are available.   info@example.com
Tel: 03-3351-5977   Fax: 03-3353-6096