[erlang-questions] Garbage Collection, BEAM memory and Erlang memory

[Roberto Ostinelli]

>
> It is only a hunch. Clearly, something is using more memory than you
> expect, and I assume you have made calculations which shows that this
> memory usage is excessive given the number of processes and the specific
> profile of the system. Running 20000 SSL sockets will have some overhead
> due to SSL, so it is important to figure out if the memory usage is normal
> or unexpected in the first place.
>

The main question here is: why is there such a difference between what the
BEAM uses, and what Erlang VM uses.

In another (bigger) test I am now seeing erlang:memory reporting a total of
10 GB, while the BEAM eats up to 15 GB.




> Also, check the operating system. If you are out of memory, and being
> killed by the OS, then there should be a log line in the kernels log about
> it. It is highly suspicious you get killed suddenly with no message at all.
> If Erlang terminates, you can arrange that it crash-dumps so you can go
> inspect the system afterwards. Alternatively, you can force it to terminate
> with a SIGUSR1 signal which will have it crash-dump. Luckily, you are in
> development mode, so being mean to the system is an option :)
>
> Other typical OS limitations are resources such as file descriptors. You
> need to verify that you are not being killed by the OS and debug from the
> bottom-up. Otherwise you will end up making a wrong conclusion somewhere
> along the way. Your goal should be to verify that what you *expect* is
> *reality*. So seek those conclusions.
>

File descriptors are ok. I have 2M available, only using 220k.



> Another common thing I do, is to run mem_sup on the machine and then add
> in an alert for processes which take up more than 5% of the memory of the
> system. If such a process occurs, you go inspect it with process_info and
> figure out if it has a rather small backtrace. In that case, you log the
> backtrace fully and if it is too long, you log only the topmost parts of
> the call chain. All of this is possible by installing a custom
> alarm_handler.
>

Doing this. No process is taking such a huge amount of memory.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150123/b79fd6b7/attachment.htm>