[erlang-questions] Logger which truncates too big terms

[Jachym Holecek]

# Robert Raschke 2011-12-02:
> On Fri, Dec 2, 2011 at 3:38 PM, Jachym Holecek <freza@REDACTED> wrote:
> > # Max Lapshin 2011-12-02:
> > > No logger can reduce messages with current state of OTP.
> >
> > Nor is it sensible logger's job to inspect or modify messages in any
> > way. Just send them to backing store with minimum hassle and make
> > best effort to minimize runtime impact on the caller. IMHO.
> >
> > >
> > https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/gen_server_ems.erl#L747
> > >
> > > this is where you need to move to protect your system from silly OOM
> > > while printing error message.
> >
> > Yep, that's the right thing to do -- message originator is responsible
> > for what the message looks like. To a logger it's just a data blob
> > decorated with originator identity, or something else telling it
> > where that message belongs. IMHO.
>
> Weeell, I think there is good cause for splitting logging into two parts:
> 1. creating log entries and 2. presenting them for human consumption.

Well, in principle that's a conceivable way to see things, however being
an old UN*X-head I'm of the firm religious belief that logs should be
ASCII text files, and I'm sure Operations would share that view. :-)

> Doing this means that you can forget about any pretty-printing of log
> messages at the point of their creation. Such, rather heavy, processing can
> be deferred to the point of actually reading the log. The latter usually
> being done off line anyway.

But how heavy it is remains under your control. Pragmatic approach to
log layout (when you can afford it) and embracing iolists from the start
lower the price. You still need to be careful and sometimes move logging
out of busy path and such, but that's a hard fact of nature the way I see
it...

Also, I prefer logs to be readily available right after the fact, already
in their final shape, and inspectable with standard tools (as imperfect as
they might be, I'd personally rather avoid reinventing them -- I miss grep
especially dearly every time I'm confronted with binary SASL logs).

> I've started doing this by only ever logging lists of terms (using the
> error_logger), always providing the module that is logging as part of this
> list. I also have a modified rb (report browser) that will call a module's
> format_log/1 function (if it exists) to take the logged list of terms and
> return a pretty string.

This introduces an opportunity for mismatch between actual log format
and what the format_log/1 callback thinks, one that will be undetected
until RB is started, no? Also, doesn't this dictate that logs only be
analysed wherever you have full code of faulty node available? Such as
*not* from your laptop in the middle of a business trip without access
to deployment/development machines? Also, how well does this work with
gigabytes of logged events when you need to analyze and fix the issue
ASAP because every minute of downtime/fault is valued at roughly your
annual salary?

I have no trouble believing logs in some binary format could be a good
choice for some particular problem domain, but I've never come across
one of those in practice...

BR,
	-- Jachym