[erlang-questions] New OTP app available at GitHub: riak_err

[Ryan Zezeski]

On Thu, Nov 11, 2010 at 1:59 PM, Scott Lystig Fritchie <
fritchie@REDACTED> wrote:

> Hi, all, there's a new OTP app available over at GitHub that other users
> may find useful: riak_err.  See https://github.com/basho/riak_err/
>
> If you've ever been bitten by having a 5MB binary in an error message
> explode (on a 64-bit platform) into over 300MB of short-term memory
> usage(*) when the error_logger tries to format the message before
> sending it to the console/TTY or writing it to a file, then you *really*
> want some kind of solution.  Answering the question, "Why did the VM
> crash with a memory allocation error?" for a grumpy customer isn't fun.
>
> If an info/error/warning message (or report) is too big(**), then
> Matthias Lang's nifty trunc_io.erl library is used to truncate the
> formatted data.
>
> The riak_err app is designed to be a mostly-painless way to limit the
> amount of RAM that error messages consume while being formatted.  It's
> just a matter of adding the riak_err application to your app's
> app-dependency list.  That list is probably already starting the 'sasl'
> app, so just add 'riak_err' immediately after 'sasl', and all should be
> well.  :-)
>
> I'd be really interested to know if there are any messages that are
> overlooked when measuring & truncating, e.g. a SASL message or
> something from the kernel or a user-generated message (using
> error_logger:info_msg() and friends).
>
> -Scott
>
> (*) This has bitten me and my colleagues several times over the last
> several years.
>
> (**) As defined by two config knobs, see the README or EDoc for details.
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>
This is funny because I was _just_ about to release an application on github
that does the exact same thing you are talking about.  I've already been
bitten a few times by this.  The method I took was to copy/paste the
standard handlers and modify them to use a function I called 'make_safe'
which would take any Erlang term and make it safe (in recursive manner) to
be formatted by io_lib.  My app then removes the standard handlers, reads
their configuration, and then adds my modified versions to the error_logger
mgr.

I'll certainly be taking a look at this ASAP.

-Ryan