[erlang-questions] Problem with Beam process not freeing memory

[Tino Breddin]

Hi,

I don't think the runtime (BEAM) is at fault here since the stable memory
usage reported by the kernel can go on for a week, depending how fast the
real memory usage catches up.

Tino


On Mon, Sep 16, 2013 at 1:51 PM, Masklinn <masklinn@REDACTED> wrote:

> On 2013-09-16, at 13:40 , Tino Breddin wrote:
> >
> > (1) Not GC'ed binaries: Could it be that binary data is not GC'ed because
> > the original dispatcher process which it was passed through before being
> > stored in an ETS table is still alive. Thus there is still some reference
> > to it? However, this would not explain why erlang:memory() reports a very
> > low value for used memory for binaries.
> >
> > (2) low-level memory leak: Some driver or NIF leaking memory, which would
> > obviously not be reported by erlang:memory(). However, then it surprises
> me
> > that the Kernel's view stays stable while the BEAM's actual memory usage
> is
> > still below the Kernel's view. It should be continuously growing in this
> > case imho.
> >
> > I'd appreciate if anyone has some more insight or experience with such a
> > behaviour, while I'm further digging into this.
>
> I have no info, but I'd suggest #3: the runtime does not release memory
> back to the OS under the assumption that the high-water mark probably
> isn't a unique event; or there is some delay between the moment where
> memory is released to the runtime and where memory is released by the
> runtime to the kernel (IIRC CPython has such a behavior with multi-stage
> allocations, internal pools and freelists[-1] and that's a fairly recent
> evolution, before 2006 CPython *never* released memory to the kernel[0]).
>
> [-1] this means memory released by a Python program may or may not be
>      immediately free()'d back to the kernel, and due to release being
>      predicated upon arena emptying[0] and ignoring freelists it will
>      not happen for all internally released memory, at least not without
>      an explicit full GC and maybe even with that
> [0] http://bugs.python.org/issue1123430
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130916/1e9a5453/attachment.htm>