Mailing List Archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[tlug] A Test Message in Japanese

BLUF: Unless TLUG is sending different copies of the messages to other
people (Mailman's not doing that because personalization is not on),
this is is a receiver-side problem for this test message, which is
trivial as far as message structure goes.  I'm pretty sure it's not a
sender-side problem in general -- but that doesn't mean TLUG can't (or
shouldn't) do something to help people with non-conforming clients.

If you're having problems with this test message, though, I'd say it's
time to upgrade or switch clients if you care about reading non-ASCII

Curt J. Sampson writes:

 > This is to test the conversions our list server is doing on
 > foreign-language messages. If you see this message in garbled form,
 > rather than a simple Japanese sentence below, please let me know.
 > (Probably a reply just to me is easier; I'll soon summarize my results
 > to the list.)

People who see it garbled should try to get the raw message (also
called "original" by some MUAs), and send the header.  Here's mine
with commentary.  If you know what you're doing you can restrict to a
few interesting headers as I do, but it's usually simpler to just send
the whole thing unless there's local information you want to hide.  I
reverse the usual convention; quoted material is *not* marked, my
comments are marked with a prefixed "* ".

* MTA fields (Return-Path and trace fields) omitted.
* These should be ignored by MUAs, or perhaps the MUA will blow up if
* DKIM authentication field omitted.  Same.

X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2HOKAAEGkRe/9pKnoJlBoI9gT4CARF?=
X-IronPort-AV: E=Sophos;i="5.70,433,1574089200"; 

* Nothing to see here except the identity of the spam filter and
  antivirus.  This is important, because many filters alter the

MIME-Version: 1.0

* conformant

Content-Disposition: inline

* bizarre but conformant.  I don't think Mailman is inserting it; it
  seems to be the sending client.

User-Agent: NeoMutt/20170113 (1.7.2)

* I remember neodogs from some SF novel or other.  conformant

X-Mailman-Version: 2.1.26
Precedence: list
List-Unsubscribe: <>,
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>,

* 8 fields of Mailman goodness, conformant

Content-Type: text/plain; charset="utf-8"

* conformant but suspect -- I wouldn't be surprised if the problem
  manifests only in a multipart content-type.

Content-Transfer-Encoding: base64

* conformant

List-Id: Tokyo Linux Users Group <>

* more Mailman goodness.  Bizarre that it follows the Content-Type and
* Content-Transfer-Encoding fields.  I guess that's Mailman's doing.
* various addressing and identification fields omitted
* I realized later that I should have reported Message-ID, mea maxima
  culpa.  Very useful for confirming you're looking at the same message.


* conformant content-transfer-encoding (length of each line is a
  multiple of 4) and the content decodes sanely

* Here's J. Hart's message, which some reported as mojibake:

X-Gm-Message-State: APjAAAUC3sGf7Vc8484Fa65VDCTCRQhZ18F98aOFVUU2TDblfEynOeTQ
X-Google-Smtp-Source: APXvYqxI+ayB16ClHas62R6UUpi2yUTUegczkoFPL0EZRKtEpngXbHzIpd1eYbbPhRboBWwLey8Njw==

* Hrm.  If I were QAnon, I'd blame Google.

User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:17.0) Gecko/20130512 Thunderbird/17.0.5

* Report User-Agent and X-Mailer fields for sure.  The composing MUA
  could be doing something weird.

MIME-Version: 1.0

* conformant

References: <>
In-Reply-To: <>

* conformant threading information

X-Mailman-Version: 2.1.26
Precedence: list
List-Unsubscribe: <>,
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>,

* conformant Mailman goodness (shouldn't affect display unless
* receiving client is very very very VERY brokked)

Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"; Format="flowed"

* Always report these when discussing encoding issues.
* conformant but bizarre.  There is zero flowed content below, and
  capitalizing the parameter tag "Format" is uh, unusual.
  format=flowed doesn't seem to be used much nowadays.  Many clients
  assume that other clients will reformat paragraphs and ignore the
  RFC's strong recommendation that lines be broken to less than 74
  halfwidth characters.  I'm not sure it's an RFC recommendation, but
  in general it's considered a bad idea to use parameters to specify
  defaults (such as "Content-Disposition: inline" for content-type
  "text" above) and features that aren't used ("format=flowed" here)
  because many clients Do Wrong Things with "rare" features.
* NB: my guess about multipart content-type was wrong.

List-Id: Tokyo Linux Users Group <>
Sender: "Tlug" <>

* Mailman goodness.
* addressing info omitted.
* Now the content:


* conformant (line lengths divisible by 4), decodes to:

looks good here.

On 02/13/2020 12:26 AM, Curt J. Sampson wrote:
> This is to test the conversions our list server is doing on
> foreign-language messages. If you see this message in garbled form,
> rather than a simple Japanese sentence below, please let me know.
> (Probably a reply just to me is easier; I'll soon summarize my results
> to the list.)
> こちらは日本語でございます。
> cjs

To unsubscribe from this mailing list,
please see the instructions at

* I have no idea where Scott's client is getting the "\#" thing from,
  and the decoded bytes in his message are clearly not UTF-8 (there
  should be no ASCII characters there at all; Japanese represented in
  UTF-8 is all 8-bit-set bytes).  Scott's client doesn't announce
  itself in the header.

* Jim Blackson writes:

I also have seen this problem. First garbled message was the 
[tlug] [announcement] Nomikai - June 14th, 2019 sent by "Justin"
<>.  The header included the lines: 
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by altair.a1d.local id x549aTJA090625

This same message received on the TLUG-Admin list had a different
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

* The difference in charsets makes no sense to me, since they're the
  same message.  The only thing I can think of is that Mailman is
  changing the charset on a plain text message when a footer is added
  (tlug does, tlug-admin does not), but as far as I can tell there's
  no need (the footer is 100% ASCII).
* X-MIME-Autoconverted is like spam- and virus-checking: it's altering
  the message content in ways which should not, but in practice could,
  confuse clients.  Mailman is not doing it, and I don't think it's a
  TLUG host (can't tell since the domain name isn't on the Internet).
  It's also possible that it does the conversions wrong.  (Believe it
  or not, there are different versions of BASE64, which use different
  punctuation characters -- 52 letters + 10 ASCII digits = 62, and you
  need 65: 64 base64 digits, and a padding character.  The padding
  character is always "=", and the MIME convention for email is that
  the other two digits are "+" and "/", but "-", "~", and/or "_" are
  used in other applications.  Eg, "/" is not file-system safe, and
  "+" has special meaning in URLs.)


Home | Main Index | Thread Index

Home Page Mailing List Linux and Japan TLUG Members Links