[erlang-questions] Memory leak related to "driver_alloc"

[Florian Odronitz]

> On 3. May 2018, at 09:38, John Högberg <john.hogberg@REDACTED> wrote:
> 
> On Wed, May 02, 2018 at 07:08:34PM +0200, Florian Odronitz wrote:
>> When I log into the node I see erlang:memory() reporting 14,6 GB of
>> memory basically all attributed to "system".
>> Digging deeper recon_alloc:memory(allocated_types) finds that
>> basically all the memory is allocated by "driver_alloc".
> 
> If you can try out OTP 21-rc1, the new memory instrumentation features
> will help you find the culprit:
> 
> http://blog.erlang.org/Memory-instrumentation-in-OTP-21/ <http://blog.erlang.org/Memory-instrumentation-in-OTP-21/>

"Those who have used erlang:memory() are probably familiar with how annoyingly general the system category can be. It’s possible to get a bit more information by using erlang:system_info({allocator, Alloc}) but the most it will do is tell you that it’s (say) driver_alloc that eats all that memory and leave you with no clue which one."

Thats my situation, exactly! Once Elixir is compatible, I will run one node under OTP 21 and then see if I can get some insight.

Thanks for helping
Florian


PS: In the meanwhile I also looked at memory fragmentation using recon_alloc:fragmentation(current).
I found some impressive carriers (>4GB) but they seem to have very good usage.

[{{:driver_alloc, 1},
  [sbcs_usage: 1.0, mbcs_usage: 0.9999680067349549, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 4937881312,
   mbcs_carriers_size: 4938039296]},
 {{:driver_alloc, 2},
  [sbcs_usage: 1.0, mbcs_usage: 0.9998221050872671, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 4819741224,
   mbcs_carriers_size: 4820598784]},
 {{:driver_alloc, 3},
  [sbcs_usage: 1.0, mbcs_usage: 0.9999122383844911, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 3822018976,
   mbcs_carriers_size: 3822354432]},
 {{:driver_alloc, 4},
  [sbcs_usage: 1.0, mbcs_usage: 0.9967773040787262, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 1326648648,
   mbcs_carriers_size: 1330937856]},
 {{:driver_alloc, 5},
  [sbcs_usage: 1.0, mbcs_usage: 0.9775849991437053, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 210428096,
   mbcs_carriers_size: 215252992]},
 {{:ll_alloc, 0},
  [sbcs_usage: 1.0, mbcs_usage: 0.88204345703125, sbcs_block_size: 0,
   sbcs_carriers_size: 0, mbcs_block_size: 34683360,
   mbcs_carriers_size: 39321600]},

 
> 
>> I don't fully understand what driver_alloc contains and what could
>> drive its growth.
>> Any thoughts?
> 
> `driver_alloc` is used for general allocations in drivers and NIFs,
> e.g. `enif_alloc()`, `erl_drv_thread_create()`, or
> `enif_inspect_iovec()`.
> 
> Regards,
> John Högberg
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180504/1da5ee69/attachment.htm>