[erlang-questions] Need help understanding why the error_logger is growing to gigabytes of memory even when idle

[Michael Truog]

If you have changed some of your busy Erlang processes to run with priority "high", that seems like it could cause the behavior you are seeing.


On 10/16/2012 07:50 PM, Kris Rasmussen wrote:
> Hi All,
>
> We are observing error_logger processes that use many gigabytes of memory over time. Unfortunately, I don't know how much time it takes for them to get into this state yet, but the memory drops to virtually nothing when we trigger a manual gc cycle. Here is some information about one such process that was using 2+GB of memory before we GC'd it:
>
> (jsweb@REDACTED)23> process_info(whereis(error_logger), [memory, heap_size, total_heap_size, messages, binary, message_queue_len, stack_size, garbage_collection]).
> [{memory,2586762616},
>  {heap_size,38263080},
>  {total_heap_size,323345205},
>  {messages,[]},
>  {binary,[{139749309541344,25907,2},
>           {139749309541344,25907,2},
>           {139749250739928,26114,2},
>           {139749250739928,26114,2},
>           {139749309588072,793,2},
>           {139749309588072,793,2},
>           {139749292946832,4927,2},
>           {139749292946832,4927,2},
>           {139749308657216,122,2},
>           {139749308657216,122,2},
>           {139749266244472,122,2},
>           {139749266244472,122,2},
>           {139749258694744,1123,2},
>           {139749258694744,1123,2},
>           {139749310288824,123,2},
>           {139749310288824,123,2},
>           {139749310160072,122,2},
>           {139749310160072,122,2},
>           {139749309587840,122,2},
>           {139749309587840,122,...},
>           {139749258436600,...},
>           {...}|...]},
>  {message_queue_len,0},
>  {stack_size,8},
>  {garbage_collection,[{min_bin_vheap_size,46368},
>                       {min_heap_size,233},
>                       {fullsweep_after,65535},
>                       {minor_gcs,3}]}]
>
> As you can see, the heap size is much smaller than the total memory. We've read elsewhere that binaries can create problems like this, but I'll admit I don't fully understand the reason. Does anyone have any idea how the error_logger could be using so much memory and what the best approach is to ensure it runs a GC cycle more often? We could simply GC the error_logger with another process more frequently, but I'd rather understand what we are doing to put it in this state first.
>
> In case it's significant, we have a custom report handler that forwards error_logger messages to log4erl.
>
> Thanks,
> Kris
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121016/7469cfeb/attachment.htm>