[erlang-questions] VM leaking memory

Frank Muller frank.muller.erl@REDACTED
Fri Feb 1 17:07:30 CET 2019

I tried two solutions to reduce the memory usage of the problematic process:

1. calling garbage:collect/0 after processing N packets (varying N=10..128).
Nothing changed at all and the bin_alloc memory stayed fragmented as you
can see:

The call to instrument:carriers/0:

The call to instrument:allocations/0:

2. Hibernating the process after processing N packets (varying N=10..128).
The HIT rate went above 90% immediately.

What is the effect of frequent hibernating on the long term? This process
is receiving about ~1200 packets/sec under normal load and can reach ~3000
packets/sec under heavy load.

Is there a better way of solving this memory issue  by tweeting the bin
allocator SBC/MBC?


A text file would work a lot better, the important information is missing
> from your last screenshot.
> This should fit binary_alloc stats for the first scheduler in a screenshot
> (your system has 48 schedulers):
> [_|[SCHED0|_]] = erlang:system_info({allocator, binary_alloc}),
> io:format("~p~n", [SCHED0]).
> On Fri, Feb 1, 2019 at 11:14 AM Frank Muller <frank.muller.erl@REDACTED>
> wrote:
>> Here is the beginning of recon_alloc:snapshot/1.
>> Yes, Lukas can help on this. VM’s memory management looks like a mystery.
>> Hi Frank,
>>> The binary_alloc seems to be the one misbehaving (not 100% sure, see
>>>> attached screenshot).
>>> I agree, it's your binary_alloc. How many single block carriers in
>>> binary_alloc do you have? What about multi-block carriers? Can you share a
>>> recon_alloc snapshot?
>>>> Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson
>>>> maybe?
>>> For us, RabbitMQ, defaulting to a lower multi-block carrier size (
>>> +MBlmbcs <http://erlang.org/doc/man/erts_alloc.html#M_lmbcs>) as well
>>> as changing the allocation strategy (+MBas
>>> <http://erlang.org/doc/man/erts_alloc.html#M_as>) made a positive
>>> difference. From the screenshot that you've shared, I am guessing that this
>>> won't work for you. This is the process that we went through at RabbitMQ:
>>> https://groups.google.com/forum/#!msg/rabbitmq-users/LSYaac9frYw/LNZDZUlrBAAJ
>>> I found Lukas' Erlang Memory Management Battle Stories from 2014 to be a
>>> great erts_alloc companion:
>>> http://www.erlang-factory.com/static/upload/media/139454517145429lukaslarsson.pdf
>>> & https://www.youtube.com/watch?v=nuCYL0X-8f4
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190201/1434e690/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: IMG_6342.jpg
Type: image/jpeg
Size: 199468 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190201/1434e690/attachment.jpg>

More information about the erlang-questions mailing list