Re: [tlug] unable to create local copy utf8 encoded Japanese MySQLdata

["Stephen J. Turnbull" <stephen@example.com>]

>>>>> "Dave" == Dave Gutteridge <dave@example.com> writes:

    Dave>     The issue isn't converting the Unicode entities (thanks
    Dave> for providing the terminology) into readable Japanese. I can
    Dave> accomplish that.

"SGML character entities" ("XML" or "HTML" is a little less accurate
but will do in appropriate contexts).

Anyway, I'll go out on a limb and say I think you're wrong.  It *is*
probably an issue of converting the character entities and other
mojibake back into Unicode.  If you can do it in the database, do it.
Have you looked at the source for the web page that is displaying
entities?  I would not be at all surprised if you see a lot of
"&amp;#19968";, ie, double encoding.

    Dave>     For the majority of Japanese text, there are four places
    Dave> I can view it:

More important to solving the problem is what are the sources?  ISTR
you copied the data from a MySQL 3.X database on the hosting service
to a MySQL 4.Y database locally, and that MySQL 3.X doesn't provide
proper support for localization and/or Unicode but 4.X does.  It's
entirely unsurprising that you're having migration problems.

    Dave>     O Via the web site on my hosting service.
    Dave>     O Via phpMyAdmin on my hosting service.
    Dave>     O Via phpMyAdmin on my local testing environment.
    Dave>     X Via the web site on my local testing environment.

Are the webservers the same?  Are the versions the same?  Are the
configured modules the same (including versions), especially PHP?  Are
the webserver configurations the same (especially default character
coding and language settings)?  You're using the same browser in each
case, right?

    Dave>     I do not understand how a simple export of the data from
    Dave> my hosting service, and import of a .sql file into my local
    Dave> environment has lost the ability to correctly display within
    Dave> a web page served from my local server.

In the context of a modern database, "a simple export" is like "an
honest politician".  If you find one, stuff it and mount it on your
wall (or donate it to a museum, which is more in keeping with the
spirit of open source!)

    Dave>     And I'm especially confused as to why it displays
    Dave> correctly in my local phpMyAdmin interface, but not my local
    Dave> web page environment.

phpMyAdmin has direct access to the database, and therefore access to
a lot more information than is provided to the browser.  I'd also bet
that phpMyAdmin has *not* been upgraded to check that data it's
getting isn't full of MySQL 3.X workarounds that 4.Y no longer uses or
understands (or perhaps now handles in a more standards-compliant way).



-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.