[erlang-questions] Logger which truncates too big terms

[Jachym Holecek]

Hi Andrew,

I didn't mean to be critical of lager in particular, just trying to
share what I've learned from some previous logging framework effort
(Hmm, I'm possibly repeating myself on that).

My views are admittedly skewed by the need to handle *huge* volumes
of traffic of almost arbitrary shapes: sometimes it's CSV for later
postprocessing, sometimes just raw terms prefixed by timestamp,
other times they're warm a fuzzy user-friendly texts etc.

Using plain iolists as message format and basically just a thin
wrapper on top of IO driver came out as natural fit for those
needs (after shooting myself in the foot endlessly for a few
weeks with excessive creativity ;-).

# Andrew Thompson 2011-12-02:
> As regards modifying log messages, lager does do a significant amount of
> that, I agree. However, I didn't do that lightly, it was done so the
> logger can't ever crash the node, and to provide more human-readable
> log messages based on the ones output from SASL, for customers using an
> erlang service who aren't erlang programmers themselves (the original
> log messages are also preserved, just in a different file).

Yep, that's a prefectly legitimate goal, and I like SASL's message
format too. I'd advise that you do all the formatting from caller's
context and stay in safe distance from gen_event, if at all possible.

But perhaps you can't do that without replacing error_logger module
first (something I've meant to do, using audit_log as backend, but
can't seem to get around to it). And perhaps you already do some of
that, I haven't studied lager's codebase in any great detail yet.
Also, perhaps your system has an upper bound on log volumes and
you know in advance you can afford centralized processing...

> Lager just meets the needs we had at basho for riak logging in
> production, there's plenty of alternatives with other design
> decisions/tradeoffs, but it's worked out very well for us so far.

Sure, can't question that. I suppose there's a religious question
of general attitude towards addressing project requirements -- my
take I suppose is (not without exceptions) to produce a minimalist
solution to the core issue, and than polish up the rough edges with
some amount of application-specific adaptation code...

BR,
	-- Jachym