[erlang-questions] Memory leak related to "driver_alloc"

[Jesper Louis Andersen]

My immediate intuition if  you have a 4GB carrier: my goodness, what are
you doing to the poor VM to get that? Putting a BluRay disc in memory?

I'd definitely try to figure out why that happens, because my intuition has
all alarms ringing.

On Fri, May 4, 2018 at 10:16 AM Florian Odronitz <florian.odronitz@REDACTED>
wrote:

>
> On 3. May 2018, at 09:38, John Högberg <john.hogberg@REDACTED> wrote:
>
> On Wed, May 02, 2018 at 07:08:34PM +0200, Florian Odronitz wrote:
>
> When I log into the node I see erlang:memory() reporting 14,6 GB of
> memory basically all attributed to "system".
> Digging deeper recon_alloc:memory(allocated_types) finds that
> basically all the memory is allocated by "driver_alloc".
>
>
> If you can try out OTP 21-rc1, the new memory instrumentation features
> will help you find the culprit:
>
> http://blog.erlang.org/Memory-instrumentation-in-OTP-21/
>
>
> "Those who have used erlang:memory() are probably familiar with how
> annoyingly general the system category can be. It’s possible to get a bit
> more information by using erlang:system_info({allocator, Alloc}) but the
> most it will do is tell you that it’s (say) driver_alloc that eats all
> that memory and leave you with no clue which one."
>
> Thats my situation, exactly! Once Elixir is compatible, I will run one
> node under OTP 21 and then see if I can get some insight.
>
> Thanks for helping
> Florian
>
>
> PS: In the meanwhile I also looked at memory fragmentation
> using recon_alloc:fragmentation(current).
> I found some impressive carriers (>4GB) but they seem to have very good
> usage.
>
> [{{:driver_alloc, 1},
>   [sbcs_usage: 1.0, mbcs_usage: 0.9999680067349549, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 4937881312,
>    mbcs_carriers_size: 4938039296]},
>  {{:driver_alloc, 2},
>   [sbcs_usage: 1.0, mbcs_usage: 0.9998221050872671, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 4819741224,
>    mbcs_carriers_size: 4820598784]},
>  {{:driver_alloc, 3},
>   [sbcs_usage: 1.0, mbcs_usage: 0.9999122383844911, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 3822018976,
>    mbcs_carriers_size: 3822354432]},
>  {{:driver_alloc, 4},
>   [sbcs_usage: 1.0, mbcs_usage: 0.9967773040787262, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 1326648648,
>    mbcs_carriers_size: 1330937856]},
>  {{:driver_alloc, 5},
>   [sbcs_usage: 1.0, mbcs_usage: 0.9775849991437053, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 210428096,
>    mbcs_carriers_size: 215252992]},
>  {{:ll_alloc, 0},
>   [sbcs_usage: 1.0, mbcs_usage: 0.88204345703125, sbcs_block_size: 0,
>    sbcs_carriers_size: 0, mbcs_block_size: 34683360,
>    mbcs_carriers_size: 39321600]},
>
>
>
>
> I don't fully understand what driver_alloc contains and what could
> drive its growth.
> Any thoughts?
>
>
> `driver_alloc` is used for general allocations in drivers and NIFs,
> e.g. `enif_alloc()`, `erl_drv_thread_create()`, or
> `enif_inspect_iovec()`.
>
> Regards,
> John Högberg
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180504/664dc08c/attachment.htm>