[erlang-questions] Any guidance on tweaking allocator settings?

[Metin Akat]

Hello,

I recently had the same problem in quite identical situation. In my case it
was a load tester application that hammered onto a REST API. With several
thousand bots when hitting a method that returns ~1M JSON, the VM started
eating huge amounts of memory, until it crashed with out-of-memory
exception.

A very simple hack solved my problem (and that being a non-critical
application I don't really care bout the implications):

[garbage_collect(Pid) || Pid <- processes()].

I execute this every minute or so and my load tester never had the problem
again.

I imagine somehow it failed to trigger full-sweep GC ever and generational
GC never cleared the old parsed JSON terms allocated in the processes'
heap. But as it solved my problem, I never dug any deeper.

Regards,
Metin



On Tue, Apr 5, 2016 at 2:58 PM, Fred Hebert <mononcqc@REDACTED> wrote:

> On 04/05, Lukas Larsson wrote:
>
>> Hello,
>>
>> If you use recon_alloc:memory(usage) on the snapshots you have you can see
>> you have a high usage percentage which means that you do not have any
>> memory fragmentation issues that tweaking the erts allocators would fix.
>> Your application is simply using a lot of memory and it seems to be
>> process
>> heap memory. I would recommend using observer to see which processes use a
>> lot of memory and from there try to figure out why.
>>
>>
> To add to this, I've got a full section on this stuff in Erlang in Anger (
> http://www.erlang-in-anger.com/). Look at Chapter 7 for memory issues,
> and 7.3 for a specific focus on allocators.
>
> That hopefully may prove helpful.
>
> Regards,
> Fred.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160407/24843316/attachment.htm>