[erlang-questions] error_logger and the perils of asynchronicity

[Per Melin]

Steve Davis:
> Hi Christian,
>
> Are you saying that the bottleneck issue with error_logger is to do with
> processing its inbox and not with the call to io:format in the receive
> that synchronously writes the message out to disk?

A flood of messages or a handful of huge SASL crash reports are
different problems.

Let's for example benchmark my little error handler that offloads the
message queue as fast as possible and does disk access in a separate
process.

When logging 100 000 one-line error messages at full speed on my
laptop it takes 5 seconds before all of them have been written to
disk. Doing the same thing to the standard error_logger_file_h takes
162 seconds.

Huge difference.

But with 20 large (~20 MB) messages both take about the same time
(~45s). Both use over 1 GB of memory in the process. Not good.

In neither case do I believe that the formatting is the real problem,
but I'm partly guessing.