[erlang-questions] Why Beam.smp crashes when memory is over?
Ulf Wiger
ulf.wiger@REDACTED
Wed Nov 11 11:38:47 CET 2009
Richard O'Keefe wrote:
>
> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote:
>
>> Richard O'Keefe wrote:
>>
>>> I am not denying the *need* for limits. (Anyone else remember "engines"
>>> in Scheme?) I've had enough functions in enough languages go into
>>> infinite recursion that I can see the point of stopping them.
>>> However, Ulf Wiger has not addressed these points:
>>> - if the tagging scheme is changed (and it has in the past),
>>> memory requirements may change.
>> > ...
>>
>> Perhaps I'm colored by having worked on systems where every
>> product release was preceded with months of relatively
>> thorough testing, where monitoring memory usage during
>> important operating conditions was routine.
>
> Let's see if we can reach consensus here.
I think we do agree. Hopefully the volley will clarify things
to the rest of the crowd - let's go another round and see if
people will start groaning. :)
> What I'm saying is that you have to *KEEP* doing it,
> as indeed you did ("every product release"), and that
> you have to keep doing it even if YOUR code didn't
> change at all.
Indeed. For high-availability products, this ought to
be a given. But as Erlang is also used in many other settings,
I would like to add an observation - perhaps obvious to all:
Erlang has some limits /today/ that programmers have to live
with. In many cases, the limits are high enough that they
are no cause for concern. Some of the better-known limits are:
- the number of simultaneous processes in the system
(default: 32768, but can be raised up to 268435456)
- the number of ETS tables (default: 1400, can be raised considerably)
- the number of open ports and file descriptors (basically
an OS limit)
Other system limits can be found in
http://www.erlang.org/doc/efficiency_guide/part_frame.html
and most are implementation details and subject to change.
What happens if you run into a system limit, e.g. if you
try creating too many ets tables, is that the operation
fails with an exception. Out of memory is a rather special
case, in that it brings down the entire VM.
Adding a few more limits that affect individual processes
rather than suffering the node-global disaster of OOM,
is thus nothing new, and in that sense shouldn't be
terribly controversial in itself. The reasonable /default/ is
of course that processes behave as today, i.e. the heap
and message queue length limits et al would be /optional/.
Having said this, it is certainly important to discuss which
such limits would actually be useful in practice.
> How you do that thorough testing is something that needs
> to be written up clearly in some new Erlang book, which
> I wish you would write. I would certainly buy it.
Ok, at least one potential customer. I'll think about it. :)
BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com
More information about the erlang-questions
mailing list