Subject: 8-bit MOO?

From: Michael P Urban <[email-masked]>
Date: Tue, 24 Nov 1992 13:25:57 -0800

I observe that MOO appears to accept only ASCII characters.  What
changes would be required for it to accept 8-bit characters, so that
someone could implement a French MOO or whatever using the ISO 8859
character set?

---

From: Pavel Curtis <[email-masked]>
Date: Tue, 24 Nov 1992 15:06:17 -0800

> Date:	Tue, 24 Nov 1992 13:25:57 -0800
> From:	Michael P Urban <[email-masked]>
> 

The server checks, on input from the network, that the characters satisfy the
`isgraph()' macro in <ctype.h>.  This test is (unfortunately duplicated) in
net_single.c and net_multi.c.

The parser/compiler and unparser for the MOO language also make some checks
that would have to be made right for a Latin-1 environment; these have to do
with the syntax of MOO identifiers.

It may be that you'd only need to do some simple little trick with ANSI C
`locales', but I've avoided knowing anything about those, so I couldn't give
you any more details.

In any case, there is nothing in the server code that assumes that the high bit
of a character is always zero or any other such nonsense, so it would be
straightforward to make it work with any 8-bit encoding.  Making it work with
multibyte characters would probably be less straightforward, but still not
particularly tough.

	Pavel

---

From: Roger Crew <[email-masked]>
Date: Tue, 24 Nov 1992 21:01:54 -0800

"
To: [email-masked]
Message-ID: <[email-masked]>

Don't the case-insensitive string comparison and search functions
(index,strsub, etc...) make assumptions about the way the character
set is laid out?

---

From: [email-masked]
Date: Tue, 24 Nov 1992 21:46:23 -0800

> (index,strsub, etc...) make assumptions about the way the character set is
> laid out?

I could be lying, but I believe that the functions toupper(), tolower(), etc.
from the ANSI, POSIX, ISO, FIPS, etc.  need to be relinked to an
internationalized libc's versions of these functions.

A good implementation of these functions and macros will use a 256 wide array
of 8 wide bit strings, each flag in each of the bit strings flagging inclusion
in a set, like numerics or capitals.  To change language locales then, all you
have to do is point the macros and functions at the new language's array.

I don't think this is really a problem.  The main concern is user code which
assumes that the 8th bit in a char is unused.  If you do that, and your
platform claims to do other languages, then you're as 8 bit clean as you can
get, theoretically.

--joe
SRI SYADASTI SYADAVAKTAVYA SYADASTI SYANNASTI SYADASTI CAVAKTAVYASCA
SYADASTI SYANNASTI SYADAVATAVYASCA SYADASTI SYANNASTI SYADAVAKTAVYASCA

WARNING: class has only private members and no friends.

---

From: Michael P Urban <[email-masked]>
Date: Wed, 25 Nov 1992 07:36:55 -0800

Your message dated: Tue, 24 Nov 92 21:46:23 PST
> internationalized libc's versions of these functions.

Well, this was an educational experience for me (so I can vaguely
justify the time I spent on it...:-) )

On HP/UX (the system I am using), functions like `toupper', the regexp
functions, the ctype functions, string collation functions, and time
conversion functions can be affected by the Native Language Support
(NLS) environment.  Adding a call to the library routine `setlocale'
will make subsequent calls to these routines behave appropriately for
the local language.  This is fairly flexible; you can set `LC_CTYPE'
behavior without setting `LC_TIME' behavior, so that `isupper' et al.
do the right things for French without causing the output from ctime
to be in French, for example.  Adding the setlocale call did not work
initially, incidentally; the calls to `isgraph' in net_multi.c and
net_single.c also had to be changed to explicitly cast their arguments
to the (correct) type int, rather than char; C programmers take note.
After that change, invoking moo with something like (setenv LANG
french; moo ...) allowed me to enter funny French characters to my
coeur's content.

For lovers of standards, The HP/UX manual page for `setlocale' has the
following information:

 STANDARDS CONFORMANCE
      setlocale: XPG3, POSIX.1, FIPS 151-1, ANSI C

I believe that more recent SunOS, uh, Solaris releases also provide
this facility, although particular installations will not have language
support for particular locales (I was sort of amazed that our rather
vanilla HP/UX system included locales like czech).

	Mike