[erlang-questions] error_logger and the perils of asynchronicity

[Steve Davis]

On May 16, 1:28 pm, Per Melin <per.me...@REDACTED> wrote:
> Steve Davis:
>
> > I'm not sure I understand your message:
>
> > Doesn't the first case suggest that it _is_ the synchronous io:format/write
> > in error_logger that's causing the backup?
>
> That's three different things: synchronous (to the inbox) processing,
> use of io:format (or io_lib:format) and writing to disk.

taking a closer look...is it really? ...from error_logger_h.erl:
    io:format(Fd, T ++ S ++ add_node("", Pid), []);
...possibly it's the ++ appends in those calls not the io:format
itself?

> My points:
> 1) error_logger could be faster for some cases without being
> synchronous for the caller.
> 2) Letting the message queue fill up in a process that is writing to
> disk is a bad idea.

agreed, and I suspect that messaging backup is being caused by the log
server process stalling on sequential calls such as the above.

> 3) io_lib:format being slow is much less of an issue here, and
> probably in most other cases that don't involve SASL error reports.
>
> > Doeen't the second case (>20Mb messages) look suspiciously like there could
> > be something like memory paging going on (i.e. something else entirely)?
>
> I focused entirely on the memory usage in the second case and didn't
> even consider that is was also slowish. That I do believe is due to
> io_lib:format. But what can one expect when pretty-printing 400 MB?

20 x 20Mb -> about 1/2 gig then a copy taken -> 1 gig
...so it doesn't enormously surprise me (except for the fact that i
understood that large binaries were explicitly NOT copied by the VM
for this very reason)

>
> > ...how much physical RAM does this laptop have? i.e. is the OS paging
> > out the memory?
>
> 4 GB. No.

fair enough, i'd still be looking for another cause though with
respect to these huge messages.

of course I could be entirely wrong about (life, the universe and)
everything, especially since i'm only been an erlanger for under a
year :) but I've not seen anything else that's convincing me as a
better explanation.

best,
/s