[erlang-questions] Erlang garbage collection

Viacheslav V. Kovalev kovyl2404@REDACTED
Sat Oct 25 20:37:03 CEST 2014

Hi all!
I need your help in investigation of issue with erlang memory 
consumption. How typical, isn't it?

We have two different deployment schemes.
* In first scheme we running many identical nodes on small virtual 
machines (in Amazon AWS),
one node per machine. Each machine has 4Gb of RAM.
* In another deployment scheme we running this nodes on big baremetal 
machines (with 64 Gb of ram),
with many nodes per machine. In this deployment nodes are isolated in 
docker containers
(with memory limit set to 4 Gb).

I've noticed, that heap of processes in dockerized nodes are hogging up 
to 3 times much more RAM, than
heaps in non-dockerized nodes with identical load. I suspect that 
garbage collection in non-dockerized nodes is more aggressive.
Unfortunately, I don't have any garbage collection statistics, but I 
wanna obtain it ASAP.

To give more information, I should say that we are using HiPE R17.1 on 
ubuntu 14.04 with stock kernel. In both schemes we are
running 8 schedulers per node, and using default `fullsweep_after` flag.

My blind suggestion is that erlang default garbage collection relies 
(somehow) on /proc/meminfo (which is not actual in dockerized environment).
I am not C-guy and not familar with emulator internals, so could someone 
point me to places in erlang sources that are responsible for garbage 
and some emulator options which I can use to tweak this behavior?

Viacheslav V. Kovalev
Software engineer, Tradingview.

More information about the erlang-questions mailing list