Encodings of 94x94-character sets
One popular way of mixing US-ASCII and a 2-byte set is to set the high
bit on the first byte of 2-byte characters.
Some schemes also set the high bit on the second byte.
An example is the scheme commonly called Shift-GB.
Note that this is quite different from
Shift-JIS (MS Kanji).
There are schemes that use ASCII sequences to switch between ASCII and
GB modes to accommodate both sets in graphic ASCII text.
For example, the HZ scheme used on the Usenet group
alt.chinese.text
wraps unshifted GB text in the strings "~{
" and "~}
".
For more details, see the
HZ package.
ISO 2022 Coding
ISO 2022:1986
ISO 7-bit and 8-bit Coded Character Sets -
Code Extension Techniques
provides several ways of encoding n-byte 94- or 96-character sets,
using escape sequences.
It uses a two-stage mapping:
- Graphic character sets are assigned to four character set slots,
called G0, G1, G2 and G3.
G0 may hold a 94-character set,
while the others may hold either a 94- or a 96-character set.
Initially G0 contains ISO 646 IRV(1983), and G1-3 are undefined.
(This set differs from US-ASCII only in that it has a currency symbol
in the place of the dollar sign; this difference is often ignored.)
These assignments may be changed with
designator sequences.
- These slots are assigned to two byte ranges:
- GL (graphic left): byte values 20-7F for 96-character sets,
21-7E for 94-character sets (in which case SPACE and DEL are defined).
- GR (graphic right): byte values A0-FF for 96-character sets,
A1-FE for 94-character sets.
Initially G0 is assigned to the GR range and G1 to the GL range.
These assignments may be changed with
shift sequences:
- A single shift changes an assignment for a single character
(one or more bytes).
- A locking shift makes a permanent assignment.
Some subset implementations are widely used.
7-bit encodings (using GL only)
- JUNET or
ISO-2022-JP
-
Designator sequences are used
to switch G0 between 94-character sets.
The guidelines explicitly require switching back to an ISO 646 variant
(such as US-ASCII or JIS-Roman)
before the end of each line.
This scheme is widely used in Japan.
- ISO-2022-JP-2
- An
extension of the previous scheme,
using designator sequences
to switch G2 between 96-character sets,
and single shifts
to include characters from these sets.
- ISO-2022-KR
- A
designator sequence is used to
assign the KSC 5601-1987 set to G1,
and locking shifts
to switch GL between G0 and G1.
- ISO-2022-INT-1
- A
mixture of the previous two.
G1 is used for KSC 5601-1987 and 96-character sets,
G0 for all other sets.
- ISO-2022-CN
- A proposed scheme combining
GB and CNS sets
using designator sequences
and locking shifts.
See the
related mailing list.
8-bit encodings (using both GL and GR)
- Extended Unix Code (EUC)
- A
locale-dependent fixed codeset assignment is assumed,
and single shifts are used
to access G2 and G3.
It is common practice to leave US-ASCII in G0.
- The X Windows
Compound Text Encoding
- Only G0 and G1 are used,
locked into GL and GR respectively.
Designator sequences
are used to switch character sets.
"Left half" sets should be used in G0 only,
"Right half" sets in G1 only,
others (including the 2-byte CJK sets) in either position.
(There is also provision for non-ISO-2022 character sets.)
For newer schemes, the convention appears to be:
- G0
- US-ASCII.
- G1
- Primary non-Latin character set
(usually a 2-byte set in the CJK context).
- G2
- Supplementary Latin-based character set
(or, in the CJK context, an auxiliary 1-byte set).
- G3
- Auxiliary non-Latin character set
(usually a 2-byte set in the CJK context).
For many mono-lingual (or mono-lingual+English) applications,
it suffices to keep a fixed configuration.
Part of Notes on CJK Character Codes and Encodings.