[erlang-questions] Need help understanding why the error_logger is growing to gigabytes of memory even when idle

[Kris Rasmussen]

Hi All,

We are observing error_logger processes that use many gigabytes of memory
over time. Unfortunately, I don't know how much time it takes for them to
get into this state yet, but the memory drops to virtually nothing when we
trigger a manual gc cycle. Here is some information about one such process
that was using 2+GB of memory before we GC'd it:

(jsweb@REDACTED)23> process_info(whereis(error_logger), [memory,
heap_size, total_heap_size, messages, binary, message_queue_len,
stack_size, garbage_collection]).
[{memory,2586762616},
 {heap_size,38263080},
 {total_heap_size,323345205},
 {messages,[]},
 {binary,[{139749309541344,25907,2},
          {139749309541344,25907,2},
          {139749250739928,26114,2},
          {139749250739928,26114,2},
          {139749309588072,793,2},
          {139749309588072,793,2},
          {139749292946832,4927,2},
          {139749292946832,4927,2},
          {139749308657216,122,2},
          {139749308657216,122,2},
          {139749266244472,122,2},
          {139749266244472,122,2},
          {139749258694744,1123,2},
          {139749258694744,1123,2},
          {139749310288824,123,2},
          {139749310288824,123,2},
          {139749310160072,122,2},
          {139749310160072,122,2},
          {139749309587840,122,2},
          {139749309587840,122,...},
          {139749258436600,...},
          {...}|...]},
 {message_queue_len,0},
 {stack_size,8},
 {garbage_collection,[{min_bin_vheap_size,46368},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,3}]}]

As you can see, the heap size is much smaller than the total memory. We've
read elsewhere that binaries can create problems like this, but I'll admit
I don't fully understand the reason. Does anyone have any idea how the
error_logger could be using so much memory and what the best approach is to
ensure it runs a GC cycle more often? We could simply GC the error_logger
with another process more frequently, but I'd rather understand what we are
doing to put it in this state first.

In case it's significant, we have a custom report handler that forwards
error_logger messages to log4erl.

Thanks,
Kris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121016/e9b2add1/attachment.htm>