[erlang-questions] Why Beam.smp crashes when memory is over?

[Michael McDaniel]

On Wed, Nov 11, 2009 at 11:38:47AM +0100, Ulf Wiger wrote:
> Richard O'Keefe wrote:
>>
>> On Nov 11, 2009, at 11:26 AM, Ulf Wiger wrote:
>>
>>> Richard O'Keefe wrote:
>>>
>>>> I am not denying the *need* for limits.  (Anyone else remember "engines"
>>>> in Scheme?)  I've had enough functions in enough languages go into
>>>> infinite recursion that I can see the point of stopping them.
>>>> However, Ulf Wiger has not addressed these points:
>>>> - if the tagging scheme is changed (and it has in the past),
>>>>   memory requirements may change.
>>> > ...
>>>
>>> Perhaps I'm colored by having worked on systems where every
>>> product release was preceded with months of relatively
>>> thorough testing, where monitoring memory usage during
>>> important operating conditions was routine.
>>
>> Let's see if we can reach consensus here.
>
> I think we do agree. Hopefully the volley will clarify things
> to the rest of the crowd - let's go another round and see if
> people will start groaning. :)
>
>> What I'm saying is that you have to *KEEP* doing it,
>> as indeed you did ("every product release"), and that
>> you have to keep doing it even if YOUR code didn't
>> change at all.
>
> Indeed. For high-availability products, this ought to
> be a given. But as Erlang is also used in many other settings,
> I would like to add an observation - perhaps obvious to all:
>
> Erlang has some limits /today/ that programmers have to live
> with. In many cases, the limits are high enough that they
> are no cause for concern. Some of the better-known limits are:
>
> - the number of simultaneous processes in the system
>   (default: 32768, but can be raised up to 268435456)
> - the number of ETS tables (default: 1400, can be raised considerably)
> - the number of open ports and file descriptors (basically
>   an OS limit)
>
> Other system limits can be found in
> http://www.erlang.org/doc/efficiency_guide/part_frame.html
> and most are implementation details and subject to change.
>
> What happens if you run into a system limit, e.g. if you
> try creating too many ets tables, is that the operation
> fails with an exception. Out of memory is a rather special
> case, in that it brings down the entire VM.
>
> Adding a few more limits that affect individual processes
> rather than suffering the node-global disaster of OOM,
> is thus nothing new, and in that sense shouldn't be
> terribly controversial in itself. The reasonable /default/ is
> of course that processes behave as today, i.e. the heap
> and message queue length limits et al would be /optional/.
>
> Having said this, it is certainly important to discuss which
> such limits would actually be useful in practice.
>
>> How you do that thorough testing is something that needs
>> to be written up clearly in some new Erlang book, which
>> I wish you would write.  I would certainly buy it.
>
> Ok, at least one potential customer. I'll think about it. :)
________________________________________________________________


   yes, please ... two customers

~Michael


>
> BR,
> Ulf W
> -- 
> Ulf Wiger
> CTO, Erlang Training & Consulting Ltd
> http://www.erlang-consulting.com
>


-- 
Michael McDaniel
Portland, Oregon, USA
http://trip.autosys.us