Subject: VM_Allocate on NeXT driving me NUTS!

From: [email-masked] (Jonathan Bradshaw)
Date: Mon, 19 Oct 1992 06:39:22 -0700

I'm running the 1.6.11 server on a NeXT 68040 slab. It has 80MB drive 
free and a 20mb swap file. The server has a 1.2MB database. On both .11
and .1 after a period of time the server will panic with a
VM_ALLOCATE error from the system. This is generally after a memory
request is made and in this case, generally large although the size may
not be key. In all cases, the full database is never dumped causing me
to lose many hours (checkpointing is at 5am-6am).

The log shows:

Oct 14 20:44:16: #2: @d #17 (#17 is the help database in core)
memory allocation error: vm_allocate failed
memory allocation error: vm_allocate failed
memory allocation error: vm_allocate failed
Oct 14 20:44:25: *** PANIC: memory allocation (size 6164) failed!
Oct 14 20:44:57: *** PANIC: memory allocation (size 100) failed!
Oct 14 20:44:57: *** RECURSIVE PANIC: aborting

Last night, at 2am it crashed again. It saved about 1MB of the database.

The log shows:

Oct 19 02:02:49: #275: "I have the dates in your email.
Oct 19 02:03:02: #275: "Do youw ant to stay at my house, beginning, middle or end? do you need ride to/from airport?
Oct 19 02:03:14: #307: "Well, Myra ...
Oct 19 02:03:23: #307: "Why don't you just ask them if it's ok to stay for a night or two?
Oct 19 02:03:24: #275: "yeah Henry?
Oct 19 02:03:25: #307: "Think small.
Oct 19 02:03:30: #307: "I think that's the way to go.
Oct 19 02:03:34: #275: "Okey doke
Oct 19 02:03:50: #275: "I dont think it will matter to them if its beginning middle or end =)
Oct 19 02:03:51: *** PANIC: memory allocation (size 32768) failed!
Oct 19 02:03:51Oct 19 02:03:51: PANIC-DUMPING on dump.PANIC ...
memory allocation error: vm_allocate failed
Oct 19 02:04:14: *** PANIC: memory allocation (size 24) failed!
Oct 19 02:04:14: *** RECURSIVE PANIC: aborting

I would REALLY appreciate some help!!!!! What can I do to stop this? I
have been downing the system at 5am and bringing it back up but this
doesn't seem to have helped.

-- 
 Jonathan Bradshaw | [email-masked] -or- [email-masked]
  Packet: n9oxe@n0ary.#nocal.ca.usa.na | Prodigy: XMSN02B | (Os/2)(DOS)(Linux)
  WNDU-AM/FM/TV South Bend, IN | Disclaimer "My opinions are not my employers"
  - "Busy life, not enough time to look in the mirror." - Deborah Ann Gibson -

---

From: Pavel Curtis <[email-masked]>
Date: Mon, 19 Oct 1992 10:31:01 -0700

> Date:	Mon, 19 Oct 1992 06:39:22 -0700
> From:	[email-masked] (Jonathan Bradshaw)
> 

First, I strongly recommend checkpointing *much* more frequently than once per
day if you can possibly afford it.

Second, to track down the memory problem, I would run the server under a
debugger, setting a breakpoint in `panic' (and also telling the debugger not to
stop when the server receives a SIGPIPE signal).  When the server next gets the
memory error, use `ps' to see how much memory it's using.  Also, you could try
using `pstat -s' before you start up the server to get an idea of just how much
swap space is left over for the server.  20 megabytes is not a great deal of
swap space for everything running on a NeXT machine to share; I wouldn't be
surprised if there was only 5 megabytes or so leftover for the server, and
that's certainly not enough.

This overall problem, of the server panicking when it runs out of memory, is
the single most severe robustness problem; unfortunately, I have not been able
to come up with any good ways to solve the problem in the current server.  It
is easier to see how one might solve it in the disk-based server, so perhaps
I'll be able to get such a fix in before that's released.

Until then, I'm sorry; you're just going to have to give it more space.

	Pavel

---

From: David Miller <[email-masked]>
Date: Mon, 19 Oct 1992 17:06:10 -0700

|This overall problem, of the server panicking when it runs out of memory, is
|the single most severe robustness problem; unfortunately, I have not been able
|to come up with any good ways to solve the problem in the current server.  It
|is easier to see how one might solve it in the disk-based server, so perhaps
|I'll be able to get such a fix in before that's released.
|
|Until then, I'm sorry; you're just going to have to give it more space.
|
|	Pavel

There is a work around for this sort of problem.  When you start up,
allocate as much memory as you will need to do a dump should a panic
occur.  When an out-of-memory panic occurs, free the saved memory and
proceed with the shutdown.  

DavidM
-- 
DavidM			CATS - Commodore Applications and Technical Support
+1 215 431 9425		[email-masked]
"Commodore Amiga 3000UX - Born to Run UNIX SVR4"

---

From: Pavel Curtis <[email-masked]>
Date: Mon, 19 Oct 1992 19:03:38 -0700

> Date:	Mon, 19 Oct 1992 17:06:10 -0700
> From:	David Miller <[email-masked]>
> 

This isn't guaranteed to work, though it does improve your chances.  The
problem is that some allocators never split an large freed objects when you ask
for smaller ones, and some allocators never coalesce small freed objects when
you ask for larger ones.  Thus, you could free up all that nice storage you
saved away and then find that malloc() still returns 0 because the memory you
freed isn't in the right size chunks.

The more difficult problem, in any case, is avoiding the out-of-memory panic in
the first place.  This is the problem that I think I may be able to solve in
the disk-based server, but not in the memory-based one.

	Pavel

---

From: [email-masked] (H. Peter Anvin N9ITP)
Date: Mon, 19 Oct 1992 19:29:09 -0700

> the disk-based server, but not in the memory-based one.

Just outof curiosity (don't take this the wrong way, please); but is there
an update on the progress of the diskbased server?  

	/hpa

-- 
INTERNET:  [email-masked]   TALK:      [email-masked]
BITNET:    HPA@NUACC     IBMNET:    16331@IBMX400
HAM RADIO: N9ITP         NeXTMAIL:  [email-masked]
rm -rf /msdos /windows /nt

---

From: Pavel Curtis <[email-masked]>
Date: Mon, 19 Oct 1992 19:42:13 -0700

> Date:	Mon, 19 Oct 1992 19:29:09 -0700
> From:	[email-masked] (H. Peter Anvin N9ITP)
> 
> Just out of curiosity (don't take this the wrong way, please); but is there

I intend to begin real work on the disk-based server sometime in the first week
of November.  Much of the design work is already done and I am already in the
process of getting the low-level database management code from researchers at
UC Berkeley.  This code, for those who understand this sort of thing, provides
an essentially DBM-like interface but uses log-based recovery techniques to
ensure transaction semantics.  [Translation: the server won't lose data when it
crashes...]

I don't know how long it will take me to finish the conversion of the server.
On the one hand, the code has already been split along the proper boundary to
be ready for swapping in a new DB representation.  On the other hand, I am also
going to be trying to work on the Jupiter (multi-media MOO) project here at
PARC, getting the first version of `easy portable window-based interfaces to
MOO objects' up and running.  (Look for a public release of the Jupiter window
system code, both in-database MOO-code and a special client program, sometime
in the first half of the new year...)

It's quite possible that I will give the Jupiter project priority over
disk-basing for much of the month of November, but I haven't decided about that
yet.  Even if I do, I would expect that we'll be testing the disk-based server
on LambdaMOO itself by mid-December, with public release by sometime around New
Year's Day.  It might come sooner, it might come later, but this is my current
best guess.

	Pavel