[erlang-questions] Need help understanding why the error_logger is growing to gigabytes of memory even when idle

[Kris Rasmussen]

Michael, thanks for the quick reply. We have not changed the priority of
our processes. It's probably also worth including this output since it
seems to imply that binaries are not the culprit as we originally suspected:

(jsweb@REDACTED)25> memory().
[{total,7382580240},
 {processes,7289606750},
 {processes_used,7289603405},
 {system,92973490},
 {atom,248121},
 {atom_used,216648},
 {binary,46419248},
 {code,5018541},
 {ets,370224}]

On Tue, Oct 16, 2012 at 7:58 PM, Michael Truog <mjtruog@REDACTED> wrote:

> **
> If you have changed some of your busy Erlang processes to run with
> priority "high", that seems like it could cause the behavior you are seeing.
>
>
>
> On 10/16/2012 07:50 PM, Kris Rasmussen wrote:
>
> Hi All,
>
>  We are observing error_logger processes that use many gigabytes of
> memory over time. Unfortunately, I don't know how much time it takes for
> them to get into this state yet, but the memory drops to virtually nothing
> when we trigger a manual gc cycle. Here is some information about one such
> process that was using 2+GB of memory before we GC'd it:
>
>  (jsweb@REDACTED)23> process_info(whereis(error_logger), [memory,
> heap_size, total_heap_size, messages, binary, message_queue_len,
> stack_size, garbage_collection]).
> [{memory,2586762616},
>  {heap_size,38263080},
>  {total_heap_size,323345205},
>  {messages,[]},
>  {binary,[{139749309541344,25907,2},
>           {139749309541344,25907,2},
>           {139749250739928,26114,2},
>           {139749250739928,26114,2},
>           {139749309588072,793,2},
>           {139749309588072,793,2},
>           {139749292946832,4927,2},
>           {139749292946832,4927,2},
>           {139749308657216,122,2},
>           {139749308657216,122,2},
>           {139749266244472,122,2},
>           {139749266244472,122,2},
>           {139749258694744,1123,2},
>           {139749258694744,1123,2},
>           {139749310288824,123,2},
>           {139749310288824,123,2},
>           {139749310160072,122,2},
>           {139749310160072,122,2},
>           {139749309587840,122,2},
>           {139749309587840,122,...},
>           {139749258436600,...},
>           {...}|...]},
>  {message_queue_len,0},
>  {stack_size,8},
>  {garbage_collection,[{min_bin_vheap_size,46368},
>                       {min_heap_size,233},
>                       {fullsweep_after,65535},
>                       {minor_gcs,3}]}]
>
>  As you can see, the heap size is much smaller than the total memory.
> We've read elsewhere that binaries can create problems like this, but I'll
> admit I don't fully understand the reason. Does anyone have any idea how
> the error_logger could be using so much memory and what the best approach
> is to ensure it runs a GC cycle more often? We could simply GC the
> error_logger with another process more frequently, but I'd rather
> understand what we are doing to put it in this state first.
>
>  In case it's significant, we have a custom report handler that forwards
> error_logger messages to log4erl.
>
>  Thanks,
> Kris
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121016/8da2a7cf/attachment.htm>