[erlang-questions] limited error_logger dump (vm crashes because of generating dump)

[Matthias Lang]

Hi,

I'm replying to an old-ish thread. Sorry, hard to keep up.

On Monday, September 20, Max Lapshin wrote:
> For several times I meet bad situation, when BEAM crashes due to
> error_logger. It begins to dump laaarge state, take much more memory
> than state was due to list nature (once, I've seen error_logger eating
> 15GB of memory) and then crashes.

You then solved it by writing io_lib_pretty_limited.erl. That's a good
solution, I've successfully used the same approach in thousands of
production systems for years:

  http://www.corelatus.com/~matthias/trunc_io.erl

As you suspected, putting a format_state function in gen_server
doesn't help much, you never know that you've taken care of every
place where something sends a huge term to the error logger.

The default error logger causes more trouble than it's worth.

(For fun: how much memory does the VM need to print a 9 megabyte list?
Ten times as much, i.e. 90 megabytes? Twenty times? Fifty times? No,
more than 100 times:

   ~ >uname -a
   Linux contorpis 2.6.32-5-amd64 #1 SMP Fri Sep 17 21:50:19 UTC 2010 x86_64 GNU/Linux
   ~ >ulimit -v 1000000     %% limit to one gigabyte
   ~ >14erl
   Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]
   
   Eshell V5.8.1  (abort with ^G)
   1> io_lib:fwrite("~p", [lists:duplicate(9000123, 34)]), ignore.   
   Crash dump was written to: erl_crash.dump
   eheap_alloc: Cannot allocate 382630800 bytes of memory (of type "heap").
)

Matt