[erlang-questions] Problem with Beam process not freeing memory

[Gustav Simonsson]

Could be the memory previously allocated by the BEAM is still counted as
VmRss after it has freed the memory back to Linux.

Comparing /proc/meminfo before and after the internal cleanup task in your
Erlang app has run might give some ideas.

Also, what happens with reported values after manually dropping the page
cache [1]?

Cheers,
Gustav

[¡] http://linux-mm.org/Drop_Caches

On Mon, Sep 16, 2013 at 1:40 PM, Tino Breddin
<tino.breddin@REDACTED>wrote:

> Hi list,
>
> I'm experiencing issues with a couple of Beam nodes where I see a huge gap
> between the node's reported memory usage and the underlying Linux Kernel's
> view.
>
> This is using R15B01.
>
> As a start an application in such a node stores a lot of tuples
> (containing atoms and binary data) in ETS tables. That proceeds until a
> point where memory usage is 70% (6GB) of the available memory. At that
> point erlang:memory() and top (or /proc/PID/status) agree roughly on the
> memory usage. Then an internal cleanup task is performed, which clears
> obsolete records from the ETS tables. Afterwards, erlang:memory() reports
> an expected low value of roughly 60MB memory usage. (This includes binary
> data). However, the kernel still reports the high memory usage values (both
> VmRss and VmTotal) for the node. The kernel's usage view will stay stable
> until the ETS tables are filled to a point where the real memory usage
> exceeds the kernel's view, then the kernel reported usage will grow as well.
>
> Now having checked the node in some details I'm wondering what causes this
> difference between the BEAM's view and the Kernel's view on memory usage. I
> have 2 ideas which I'm checking right now.
>
> (1) Not GC'ed binaries: Could it be that binary data is not GC'ed because
> the original dispatcher process which it was passed through before being
> stored in an ETS table is still alive. Thus there is still some reference
> to it? However, this would not explain why erlang:memory() reports a very
> low value for used memory for binaries.
>
> (2) low-level memory leak: Some driver or NIF leaking memory, which would
> obviously not be reported by erlang:memory(). However, then it surprises me
> that the Kernel's view stays stable while the BEAM's actual memory usage is
> still below the Kernel's view. It should be continuously growing in this
> case imho.
>
> I'd appreciate if anyone has some more insight or experience with such a
> behaviour, while I'm further digging into this.
>
> Cheers,
> Tino
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130916/e6c4832c/attachment.htm>