[erlang-questions] Why Beam.smp crashes when memory is over?

[Angel Alvarez]

El Martes, 10 de Noviembre de 2009 05:36:44 Richard O'Keefe escribió:
> 
> On Nov 9, 2009, at 10:53 PM, Kenneth Lundin wrote:
> > What we could do is to make it easier for the user to prevent OOM
> > situations and also to
> > let him take the decision when it occurs or rather before it occurs.
> >
> > One way would be to let the user set a memory quota on a process with
> > options at spawn time. When the process reaches it quota it can be
> > automatically killed or the user can
> > be notified in some way and take actions.
> 
> One of the reasons this hasn't been done is, I presume, the fact that
> it is quite difficult for a programmer to determine what the memory
> quota should be.  It depends on
>   - the nature of the code being run
>   - the data it is processing (data loads tend to increase with the  
> years)
>   - the cleverness of the compiler
>   - the data representation used by the Erlang system (e.g., whether
>     pointers are 32 bits or 64, whether there are special tags for small
>     arrays
>   - the data representation used by library modules the code calls
>     (change the representation of ordered sets and a bound that might
>     have worked might not any more)
>   - a system policy on whether it's better to be tight on memory and
>     tolerate some processes crashing and needing to be restarted or
>     whether it's best to keep more processing running at the cost of
>     using more memory, a thing that might change at run time.
> 
> I have long bemoaned the (measured!) fact that the size of a stack
> frame in C can vary by a factor of 10, so that determining the sizes
> for POSIX thread stacks is a game of Russian Roulette.
> 
> You can determine suitable sizes for a particular release by running
> experiments, but the sizes thus determined are ONLY reliable for the
> specific release or releases you tried on the machine or machines
> that you tried it or them on.  Set a limit, and you can *EXPECT*
> working processes to be crashed as a *NORMAL* outcome.
> 
> Once some feature goes in, I expect to see messages in this mailing
> list "My program gets lots of killed processes due to Out-Of-Memory
> but it used to work and I haven't changed it."
I agree with You Richard

Most of our production java code throws millions of nullpointer exceptions messages
and management staff, and developers dont care, if users are happy.

I presume many subtle errors came from those nullpointer errors but people got used
to have those messages when the moved from other platforms to java and now is 
the norm among java developers.

Its not acceptable for a platform that focuses on highly scalable, survivable and error tolerant
systems that programmers were acustommed to see a lot of memory errors.

Everyone here knows how to measure memory available at program life-time (im new but, i think i knows ), there is no need
to cover such poor memory management with artificial limits, this is no C and the like.

Virtual memory and Garbage collection should garantee enough memory provided the programmers know something about the algorithms
and data structures (good unit testing).

Imagine someone asks a question on the list, provides the offending code and everyother who is helping sees lot of memory errors
on his code..

Should i waste my time helping someone when his code spurts so much memory errors and hi/her doesnt care?

Should newbies like me be exposed to such poor desing and coding style?

What's better? counting bits properly or just set the limit and wait the process for a memory crash??

How do you manage diferent compilers code size, or 32/64bits overhead? IMHO its up to the programmer concise 
memory usage expectation and this work has to be done on desing fase, and not let some black magic, i pressume 
this wil not be portable across many erlang versions and setups...

Clearly for memory battles on process/threads and the like we have already many good languages like C
despite most people here seems to want the VM to became a better OS...

Classic behavior (let the VM crash) should be preserved for legacy systems and for academic and marketing stuff...

I agree, Shielding the VM againts many error condictions and you probably will generate as much poor programmers.


/Angel

PS: I agree with Joe, Erlang best feature is a new mental model of dealing with faulty conditions more than the "multicore" promises 
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
> 
> 



-- 
Este correo no tiene dibujos. Las formas extrañas en la pantalla son letras.
__________________________________________

Clist UAH a.k.a Angel
__________________________________________
HONEY BUNNY - Any of you *uckin' pricks move and I'll execute every mother*ucking last one of you.