Subject: more on ISO 10646/Unicode debate

From: Dorothy Day <[email-masked]>
Date: Sat, 16 Feb 91 22:12:00 EST

Since there has been some discussion on this list about the merits of
Unicode, I'm forwarding this critique of Unicode by Harry Gaylord. The
Text Encoding Initiative working committees have struggled with much
broader questions of text markup (including syntactic markings and
much more), but the estabishment of a standard of character coding for
electronic transmission of material in all languages is still
fundamental to all other types of encoding.
        --Dorothy Day

--------------------------------------------------------------------------

Date:         Fri, 15 Feb 91 15:45:20 MET
Reply-To:     Text Encoding Initiative public discussion list
              <[email-masked]>
From:         Harry Gaylord <[email-masked]>
Subject:      Unicode

I have been asked by several people to say something about the implications
of the arrival of Unicode for TEI. Several useful comments in general have
appeared on Humanist, TEI-L, and 10646 about relevant issues. Yet it is
difficult to say anything succintly at this point.
  One thing is clear. No character set so far has tackled the problem of
the need to encode the lang characteristic in texts. This was already
pointed out in P1 and elsewhere. This, it seems to me, is very important
regardless of which coded character set one uses.
  There are advantages and disadvantages to both Unicode and ISO 10646 as
they are currently formulated. Hopefully they will be merged into one
ISO standard. There is no need for two multi-byte standards to be used in
different systems or even worse in single systems.
  Unicode and 10646 and the 8859 family of coded character sets have a
different understanding of what a character is how it will be used. Unicode
says nothing about the imaging of texts on a screen or printing on paper.
In a Unicode file the Greek letter alpha + IOTA SUBSCRIPT + ROUGH BREATHING
MARK + GRAVE ACCENT would be coded in 4 16-bit bytes. The software used to
image this text would have to recognize this combination of one spacing and
three non-spacing characters and put the image on your screen. ISO 10646
and the 8859 family the approach has been to have each combination as a
different coded character. Therefore this combination would be one byte in
10646. This would be a 32-bit byte if one were using the full 10646 set
or possibly 16 or 8 bits if one were using one of the compression techniques.
The software running the system with Unicode would also have to know that
since there are two accents above they have to be located differently above
the letter than if there is only one.
  On the other hand some languages have so many different combinations that
it is common practice to use "floating accents" or graphic character
combination encoding. An example of this is Hebrew which has 23 consonants
and 5 final forms. Its vowels and other signs are imaged in relation to the
consonants. If one had a coded character for each possible combination, it
would be enormous. Therefore present systems, e.g. Nota Bene SLS, and others
encode these separately. This is also true of Unicode and 10646. It is
uneconomical to do it otherwise.
  Two basic criticisms of the present proposals in 10646 are the very large
number of wasted control character positions in it, and inadequate provision
for graphic character combination encoding. In the latter there is an
appendix referring to they way this can be done under another ISO standard,
but this appendix is not a required part of the standard itself.
  The TG on character sets is in contact with Unicode and ISO with our
concerns for their work.
  We must remember that the final outcome of what is delivered is still very
uncertain. The standards have to be formulated and then hardware manufacturers
have to be convinced of the importance of them and implement them. This all
takes time. It is also important to note that the big players have people
working in the Unicode consortium and the ISO 10646 committee.
  One concern that I have is the need for representing text as it is contained
in older books and manuscripts. Neither standard as far as I can see has the
long s of English printing in earlier books. Yet we need it for many scholarly
purposes. From the standpoint of both of these standards it would be classified
as a "presentational variant" of s and be placed in a completely different
section of the character set. This is even more true of letter shapes as they
appear in manuscripts.
  There is room in each proposal for private use characters which can be used
by agreement of two or more parties. Yet the more that is included in a
standard as standard, the better off we are.
  There are currently attempts to combine the work of the Unicode consortium
and the committee for 10646. Let's hope they are successful and that the
results improve on both.
Harry Gaylord

>From [email-masked] Tue Jul  9 13:49:27 1991
Date:         Tue, 9 Jul 1991 15:26:55 MET
Reply-To: Undetermined origin c/o Postmaster <[email-masked]>
Sender: Chinese Computing Network <[email-masked]>
Comments:     <Parser> E: Mail origin cannot be determined.
Comments:     <Parser> E: Original tag was -> FROM: Jian Luo
              ([email-masked]  +31-15786227)
From: Undetermined origin c/o Postmaster <[email-masked]>
To: Multiple recipients of list CCNET-L <CCNET-L@UGA>
Status: RO

There is an interesting article published on the newest Byte (version of
July 1991) about Unicode and ISO 10646.  The title is "ASCII Goes Global",
authored by K.M. Sheldon, from page 108 through 116.

Actually it's only an introduction and it doesn't require an expert on coding
to understand it.  It also tells the different opinions from the Chinese,
Japanese and Korean sides about the coding standardization proposals.

Isn't this an interesting sentence: "Cultural issues are particularly
sensitive in the Far East; for example, the Japanese standards process is
driven in part by the traditionally conservative Ministry of Culture"?

Good day!

>From [email-masked] Tue Jul  9 14:06:57 1991
Date:         Tue, 9 Jul 1991 14:00:51 EDT
Reply-To: Scott Horne <[email-masked]>
Sender: Chinese Computing Network <[email-masked]>
From: Scott Horne <[email-masked]>
Subject:      Unicode
To: Multiple recipients of list CCNET-L <CCNET-L@UGA>
Status: RO

It's true.  Japan's Ministry of Education (Monbusho^) is trying to keep the
set of Japanese _kanji_ contiguous within Unicode.  This is wasteful and
 foolish,
of course, but guided by ultra-conservatism.

                                        --Scott