[erlang-questions] eheap_alloc crash when using mnesia

[Thomas Lindgren]

--- On Tue, 10/21/08, Robert Raschke <rtrlists@REDACTED> wrote:

> From: Robert Raschke <rtrlists@REDACTED>
> Subject: Re: [erlang-questions] eheap_alloc crash when using mnesia
> To: erlang-questions@REDACTED
> Date: Tuesday, October 21, 2008, 10:07 AM
> On Mon, Oct 20, 2008 at 9:56 PM, Matthew Dempsky
> <matthew@REDACTED> wrote:
> > I'd love to see a more elegant solution for
> out-of-memory conditions,
> > but it seems difficult to recover from without
> explicit memory
> > allocation.  E.g., in C, you can check whether
> malloc(3) failed and
> > then fail gracefully, making sure your error handling
> code doesn't
> > need to allocate additional memory.  (Stack space is
> still implicitly
> > allocated, but I've never heard this to be a
> problem, except with
> > runaway recursive programs and kernel programming
> where the stack
> > space is more limited.)
> 
> Beware! With modern kernels it is no longer true that
> malloc always
> fails if you run out of memory. Often you'll find it
> crashes upon
> access of the allocated memory rather than on the call to
> malloc().
> I'll need to dig out the references.

Sounds pretty painful; I've mostly heard complaints about the Linux OOM killer so far.

For Erlang, we could use some of the following (perhaps configurable per process):

1. Kill processes that use more than a predefined amount of memory.
2. Kill processes that spawn more than some limit.
3. Kill unimportant processes.
4. Kill greedy supervisors
5. Let the emulator die, have heart restart it and exercise our smooth failover code. (What we do today.)

Of course, just killing processes still leaves the various tables around, which can eat a considerable share by themselves (and are natural sources of space leaks to boot; doubly so if you start killing processes that would eventually have removed some entries). 

Anyone with a good solution to killing tables and/or table entries? :-)

Best,
Thomas



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com