Encodings of 94x94-character sets

One popular way of mixing US-ASCII and a 2-byte set is to set the high bit on the first byte of 2-byte characters. Some schemes also set the high bit on the second byte. An example is the scheme commonly called Shift-GB. Note that this is quite different from Shift-JIS (MS Kanji).

There are schemes that use ASCII sequences to switch between ASCII and GB modes to accommodate both sets in graphic ASCII text. For example, the HZ scheme used on the Usenet group alt.chinese.text wraps unshifted GB text in the strings "~{" and "~}". For more details, see the HZ package.

ISO 2022 Coding

ISO 2022:1986 ISO 7-bit and 8-bit Coded Character Sets - Code Extension Techniques provides several ways of encoding n-byte 94- or 96-character sets, using escape sequences. It uses a two-stage mapping:

  1. Graphic character sets are assigned to four character set slots, called G0, G1, G2 and G3. G0 may hold a 94-character set, while the others may hold either a 94- or a 96-character set.

    Initially G0 contains ISO 646 IRV(1983), and G1-3 are undefined. (This set differs from US-ASCII only in that it has a currency symbol in the place of the dollar sign; this difference is often ignored.) These assignments may be changed with designator sequences.

  2. These slots are assigned to two byte ranges:

    Initially G0 is assigned to the GR range and G1 to the GL range. These assignments may be changed with shift sequences:

Some subset implementations are widely used.

7-bit encodings (using GL only)

JUNET or ISO-2022-JP
Designator sequences are used to switch G0 between 94-character sets. The guidelines explicitly require switching back to an ISO 646 variant (such as US-ASCII or JIS-Roman) before the end of each line. This scheme is widely used in Japan.
ISO-2022-JP-2
An extension of the previous scheme, using designator sequences to switch G2 between 96-character sets, and single shifts to include characters from these sets.
ISO-2022-KR
A designator sequence is used to assign the KSC 5601-1987 set to G1, and locking shifts to switch GL between G0 and G1.
ISO-2022-INT-1
A mixture of the previous two. G1 is used for KSC 5601-1987 and 96-character sets, G0 for all other sets.
ISO-2022-CN
A proposed scheme combining GB and CNS sets using designator sequences and locking shifts. See the related mailing list.

8-bit encodings (using both GL and GR)

Extended Unix Code (EUC)
A locale-dependent fixed codeset assignment is assumed, and single shifts are used to access G2 and G3. It is common practice to leave US-ASCII in G0.
The X Windows Compound Text Encoding
Only G0 and G1 are used, locked into GL and GR respectively. Designator sequences are used to switch character sets. "Left half" sets should be used in G0 only, "Right half" sets in G1 only, others (including the 2-byte CJK sets) in either position. (There is also provision for non-ISO-2022 character sets.)

For newer schemes, the convention appears to be:

G0
US-ASCII.
G1
Primary non-Latin character set (usually a 2-byte set in the CJK context).
G2
Supplementary Latin-based character set (or, in the CJK context, an auxiliary 1-byte set).
G3
Auxiliary non-Latin character set (usually a 2-byte set in the CJK context).

For many mono-lingual (or mono-lingual+English) applications, it suffices to keep a fixed configuration.


Part of Notes on CJK Character Codes and Encodings.