[erlang-questions] VM silently exits

[Steve Vinoski]

2010/7/29 Björn Gustavsson <bgustavsson@REDACTED>:
> On Wed, Jul 28, 2010 at 10:29 PM, Steve Vinoski <vinoski@REDACTED> wrote:
>> I've found a few places in the VM C source code where exit() is called
>> without logging anything. Some of these are normal exits, like when
>> you exit an Erlang shell, where no logging is needed. But others seem
>> to be error conditions, and there should be logging for those. I think
>> I'll probably have to patch my system to add logging to those cases to
>> try to track down this problem -- is there still time to get a patch
>> like this into R14B?
>
> Yes, there should still be time. The issue seems important enough
> to fix in R14B.

Just to close off this thread:

1. I went through the exit() calls again, more carefully this time,
and it looks like the ones that need logging already have it. So, I
was wrong, no patch needed.

2. I found that the VM was dying due to running out of memory, and
that nothing was reported because heart is aggressive in restarting
the VM (as it should be). It uses SIGKILL, so the VM often has no
chance to log anything, emit a crash dump, etc.

To track down my issue I temporarily modified heart to first issue a
SIGUSR1, thereby initiating a crash dump, then waiting 10 seconds
before issuing a SIGQUIT if needed to cause a core dump, and after a 1
second sleep finally issuing a SIGKILL if needed to kill off any
non-responsive VMs. Normally you don't want heart waiting 10-15
seconds like this before restarting, since you just want the VM to go
back into action as quickly as possible, but the temporary change was
handy for debug and allowed me to successfully track down the source
of the problem (which turned out to just be unchecked growth on an ets
table, doh).

--steve