Subject: RE:A general query for the Moo folks...

From: [email-masked]
Date: Fri, 2 Apr 1993 06:42:29 -0800

>Had there been any port of the MOO server code to the Macintosh systems?
>I know they're just PC's, and I'm likely to get flammed, but it's what I
>have access to and would like to see it.

The line continues to blur between "just PC's" and workstations, and I did
manage to build 1.7.0 on the Mac, single-user. Unfortunately, that habit
the MOO has of keeping the database in RAM means big trouble for PCs, which
are still trying to get used to the idea of virtual memory. I don't have
a machine with enough RAM (virtual or otherwise) to load the LambdaCore,
and minimal.db is no fun, no fun at all.

My solution? Well, I've tried to hack diskbasing into 1.7.0, with some luck.
The thing is currently using the resource manager as it's DBMS, slow but
easy. I'm able to transform LambdaCore into resources and then swap objects
in and out as required, but there's still a massive memory leak somewhere
that has prevented me from actually being able to use the core.

It was obvious that I was able to get as far as I had because of work Pavel
had done towards diskbasing. I am unsure of the many assumptions I had to
make. For this system to work, for example, "retrieve" must always be called
to obtain a valid object pointer. Also, any call to "retrieve" might result
in the out-swapping of any other object, so that after executing the following
snippet:
  x = retrieve(objid);
  y = retrieve(objid+1);

only y could be known to contain a valid object pointer. With the continued
imminence of Pavel's disk-based server, I decided to set this aside and wait,
frustrating though it is to be so close, yet so far away. Pavel, can you offer
any encouragement either along the lines of "you're close" or "give up and
wait?" Either would be nice to hear.

So, the short of it is... I can build a non-diskbased MOO server for the Mac,
but it won't do you much good unless you have vast tracts of RAM to play with.
Believe me, you'd spend less time getting frustrated with minimal.db than I'd
spend recompiling the server :).
  -James/Minstrel-

/-----------------------------------------------------------------------------\
| James Powell - Software Engineer? - Mac/UNIX/VR/RDBMS - [email-masked]  |
|                   "No matter where you go, there you are."                  |
\-----------------------------------------------------------------------------/

---

From: Pavel Curtis <[email-masked]>
Date: Wed, 14 Apr 1993 11:08:35 -0700

> Date:	Fri, 2 Apr 1993 06:42:29 -0800
> From:	[email-masked]
> 
> Also, any call to "retrieve" might result in the out-swapping of any other
> object, so that after executing the following snippet:
>   x = retrieve(objid);
>   y = retrieve(objid+1);
> only y could be known to contain a valid object pointer.

This is almost certainly a disaster.

> Pavel, can you offer any encouragement either along the lines of "you're
> close" or "give up and wait?" Either would be nice to hear.

Give up and wait.  Sorry!

	Pavel

---

From: Greg Hudson <[email-masked]>
Date: Wed, 14 Apr 1993 12:00:30 -0700

> Pavel, can you offer any encouragement either along the lines of
> "you're close" or "give up and wait?" Either would be nice to hear.

I happen to have been looking at Marcus Ranum's cache code recently,
and I think I've worked out how I'm going to deal with caching in
Coldmud.  I'll discuss mjr's cache code first, and then how I plan to
revise it.  The ideas I discuss should be just as applicable to MOO as
they are to Coldmud.

The code I'm going to discuss is copyright 1991 by Marcus Ranum, and
so might be any derivative works.  On the other hand, the code does
get used in Tinymush and Coolmud and is part of the
freely-distributable Untermud; as far as I know, mjr doesn't mind
people using it.  He can be contacted at [email-masked].

The basic philosophy of mjr's code is that you have 'active' and
'inactive' chains of object holders.  (The number of active and
inactive chains is called the 'cache width', and you select a chain by
hashing the object ID.)  The active chain holds objects that the cache
believes to be important to your program; they will not be swapped out
under any circumstances.  The inactive chain may contain some empty
holders, as well as objects that the cache code believes not to be
important to your program.  These objects can be swapped out (using a
'dirty' bit to keep track of whether they have to be written or just
forgotten about).

At any point, your program can tell the cache code that it doesn't
care about any objects ("resetting the cache"), at which point all
containers in the active chains are moved to the inactive chains.
>From that point until the next time the cache is reset, the cache
moves requested objects onto the active chains from the inactive
chains or from disk.  In order to keep the size of the cache
relatively constant, constant, the cache reuses inactive object
holders when it reads an object in from the disk, swapping out any
object the recycled holder might have contained.  If there are no
inactive object holders available on the appropriate inactive chain,
then the cache grows in size.

Alright, that's mjr's code, minus a few details.  The problem with
this code is that at some point during the uptime of the program, the
cache is likely to blow up and load most of the database into memory.

This can happen in two cases in Coldmud: first, because of the
multiple-inheritance scheme I'm using (which I discussed in previous
messages to this list), parts of my code have to traverse the children
digraph of an object in order to update their ancestor precedence
lists.  If the object is the root object, or, say, 'Generic Described
Object', then mjr's code would load in most of the database.

Second, and independent of Coldmud's LRUJ inheritance scheme, it's
going to be fairly common that Joe Average Programmer decides to call
a method on a whole slew of objects.  (For that matter, if your core
is badly designed, Joe Average User might manage it as well.)  At no
point during interpretation of a task can Coldmud release all of the
objects it's holding onto; unless Coldmud can release objects one at a
time, the cache will load each visited object into memory, and not
release it until the task is done.

I plan to use a scheme conceptually very much like mjr's but using
reference-counting instead of an overall cache reset.  Retrieving an
object already stored in an active chain ups its reference count;
retrieving an object not stored in an active chain installs it in the
appropriate active chain with a reference count of 1.  Any piece of
code that retrieves an object is responsible for releasing it, which
decreases the reference count of the object in the active chain and,
if the reference count hits 0, moves it to the inactive chain (where
it may be swapped out on a later retrieve).

By storing the cache data in the objects themselves, rather than using
object holders, I can avoid having to do a search through the active
chains on a release, at the expense of easy reusability.

--GBH

---

From: [email-masked]
Date: Fri, 2 Apr 1993 13:51:00 -0800

>This is almost certainly a disaster.

Yeah, I knew this'd take some more work. Is my first assumption (that retrieve
is always used to fetch an object pointer) sound? Also, is there a clean
way to free all memory allocated to an object (it's name, verbdefs, propdefs,
propval, and the object itself, and whatever else)? I'm trying to do it
manually now, and I'm afraid I'm missing something. I know I said I wasn't
going to work on this, but I'm still damnably curious.

>> close" or "give up and wait?" Either would be nice to hear.

Thanks to Greg Hudson and Jeff for the reviews of other approaches to disk-
basing. Besides pointing us to other implementations, your descriptions help
highlight the problems with diskbasing. I hope we get to see a portable MOO
implementation soon.
  -James-