iconv on Solaris

Mon Sep 24 05:02:00 GMT 2001

On 2001年9月24日, Jeff Sturm wrote:
> Apparently Solaris 8 iconv supplies the missing converters but breaks
> somewhere else.

Or not. I just checked more carefully. 0xFEFF is the Byte Order Mark.
>From unicode.org:

Q: What is a BOM?
A: The special characteristics of U+FEFF ZERO WIDTH NON-BREAKING SPACE
have been provided for use by higher level protocols as a signature at the
beginning of certain data streams (primarily unmarked plaintext files).
(When used as a signature, it is referred to as the BOM character, for
Byte Order Mark.) Under those protocols, the BOM may be mandatory in those
Unicode data streams.
In the absence of such protocols and when not at the beginning of a text
stream, U+FEFF is given its normal interpretation, as ZERO WIDTH
NON-BREAKING SPACE, and is part of the content of the file or string. [MD]
Jeff