[erlang-questions] error_logger and the perils of asynchronicity

[Per Melin]

Marc Sugiyama:
> In thinking specifically about error_logger, one solution might be to
> have it check its queue and shed work when it is overloaded.  For
> example, it could shed work by:
>
> 1. dumping unformatted messages into the log rather than formatted messages.
> 2. abandoning a message and noting in the log that it lost a message.
> 3. abandoning messages with no record.
>
> More generally, some library support for detecting overloaded
> processes might be helpful.  I suspect there are several strategies
> for doing so (e.g., checking the message queue length. time to process
> the message queue, etc.).

I'm working on a handler that plugs into error_logger but circumvents
some of the issues I've had with error_logger, and also adds some
features that I really miss (like date-based log file rotation).

I use strategy 2 from your list, throwing away messages but noting it
in the log once it catches up. My problem is finding reliable
strategies for detecting that we are falling behind.

I do the writing to disk in a separate process, and detect flooding by
keeping a count of queued up messages waiting to be written. Looking
at the size, instead of the length, of the queue would be much better,
but I don't know any (fast) way to do that.