[erlang-questions] GC memory consumption
Fri Jun 24 05:49:36 CEST 2011
Erlang application we develop encountered memory lack problem (It's
32-bit unix system, so memory is limited by 2GB total per application).
Analysing crash dumps I have found consistent scenario for the issue.
1. Slogan: eheap_alloc: Cannot allocate 298930300 bytes of memory (of
2. All the processes are in 'waiting' state except one which is in
3. The 'Garbing' process 'Stack+heap' is 59786060 bytes, OldHeap is 0
bytes, some fragment data available (174663 bytes or so).
4. System has also pretty large mnesia tables, this explains why there
is not enough memory to allocate 300 MB segment at some point of execution.
Looking at the GC code I found that it always allocates new heap segment
to do its work. So in our consistent pattern GC tries to allocate
298930300 bytes to perform garbage collection for the process which heap
is 59786060 bytes.
Looks ridiculous - to de-fragment ~60 MB of the process heap, GC needs
additional ~300 MB segment... This makes very difficult to design
processes which have to keep large amount of data it its space (for
example big tree-like structure or in-memory graph).
Is there any known workaround for this (except moving it to 64 bit OS) ?
More information about the erlang-questions