[erlang-questions] Out of memory and garbage collection

[Jeroen Koops]

Interesting, and consistent with what I'm seeing. I have an application that
spawns a few thousand gen_servers. Each server instance initializes itself
with a couple of mnesia queries, which may produce a lot of temporary data.
The data that is eventually used as the gen_server's state is very limited,
however.

In my test-setup, after initialization, the gen_server's just sit idle, and
indeed seem to hang on to all heap-memory needed during initialization,
leading to an out-of-memory error after creating only a few hundred
processes. This is solved by forcing a garbage collection right after
initialization, and probably also by using the fullsweep_after spawn option
(which I still have to look into).

Thanks!

On Thu, Jan 22, 2009 at 1:33 PM, Christian <chsu79@REDACTED> wrote:

> 2009/1/22 Jeroen Koops <koops.j@REDACTED>:
> > Hi,
> >
> > I have a simple question: In Java, it is guaranteed that, before an
> > OutOfMemoryException it thrown, all possible garbage collection has been
> > done. So, out of memory really means out of memory.
> >
> > Does a similar guarantee hold for Erlang as well? Or is it possible for
> the
> > system to terminate with an out of memory error while garbage collection
> > could have freed more memory?
>
> The later. This article could probably be worth to read:
>
>
> http://www.lshift.net/blog/2008/10/31/erlangs-gc-only-runs-when-youre-running
>
> This led to the strategy: "This sounds reasonable - let's try
> hibernating after every message and see what happens." in this
> article:
>
>
> http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-2/
>
> Garbage collection is per-process in the default memory model. As a
> strategy, long running processes should probably do only little work
> so they can be hibernated, and frequent work should probably be done
> in short lived processes so all resources are freed instantly when
> they die.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090122/bfa5373a/attachment.htm>