[erlang-questions] Why Beam.smp crashes when memory is over?

Ulf Wiger ulf.wiger@REDACTED
Wed Nov 11 11:38:47 CET 2009

Richard O'Keefe wrote:
> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote:
>> Richard O'Keefe wrote:
>>> I am not denying the *need* for limits.  (Anyone else remember "engines"
>>> in Scheme?)  I've had enough functions in enough languages go into
>>> infinite recursion that I can see the point of stopping them.
>>> However, Ulf Wiger has not addressed these points:
>>> - if the tagging scheme is changed (and it has in the past),
>>>   memory requirements may change.
>> > ...
>> Perhaps I'm colored by having worked on systems where every
>> product release was preceded with months of relatively
>> thorough testing, where monitoring memory usage during
>> important operating conditions was routine.
> Let's see if we can reach consensus here.

I think we do agree. Hopefully the volley will clarify things
to the rest of the crowd - let's go another round and see if
people will start groaning. :)

> What I'm saying is that you have to *KEEP* doing it,
> as indeed you did ("every product release"), and that
> you have to keep doing it even if YOUR code didn't
> change at all.

Indeed. For high-availability products, this ought to
be a given. But as Erlang is also used in many other settings,
I would like to add an observation - perhaps obvious to all:

Erlang has some limits /today/ that programmers have to live
with. In many cases, the limits are high enough that they
are no cause for concern. Some of the better-known limits are:

- the number of simultaneous processes in the system
   (default: 32768, but can be raised up to 268435456)
- the number of ETS tables (default: 1400, can be raised considerably)
- the number of open ports and file descriptors (basically
   an OS limit)

Other system limits can be found in
and most are implementation details and subject to change.

What happens if you run into a system limit, e.g. if you
try creating too many ets tables, is that the operation
fails with an exception. Out of memory is a rather special
case, in that it brings down the entire VM.

Adding a few more limits that affect individual processes
rather than suffering the node-global disaster of OOM,
is thus nothing new, and in that sense shouldn't be
terribly controversial in itself. The reasonable /default/ is
of course that processes behave as today, i.e. the heap
and message queue length limits et al would be /optional/.

Having said this, it is certainly important to discuss which
such limits would actually be useful in practice.

> How you do that thorough testing is something that needs
> to be written up clearly in some new Erlang book, which
> I wish you would write.  I would certainly buy it.

Ok, at least one potential customer. I'll think about it. :)

Ulf W
Ulf Wiger
CTO, Erlang Training & Consulting Ltd

More information about the erlang-questions mailing list