<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">Thank you Lukas & Sverker. Patiently waiting for the next Erlang VM segfault : )</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 16, 2017 at 4:10 PM, Lukas Larsson <span dir="ltr"><<a href="mailto:lukas@erlang.org" target="_blank">lukas@erlang.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">You need to instruct the operating system to save a core dump, <a href="http://man7.org/linux/man-pages/man5/core.5.html" target="_blank">http://man7.org/linux/man-<wbr>pages/man5/core.5.html</a>.<div><br></div><div>Some operating systems can be configured to dump parts of the core dump to the syslog when a segfault happens, which can be useful but a full core dump contains much more information so is a lot more useful.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Lukas</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 16, 2017 at 5:00 PM, Gerhard Lazu <span dir="ltr"><<a href="mailto:gerhard@lazu.co.uk" target="_blank">gerhard@lazu.co.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">The crash dump was never generated because the Erlang VM segfaulted.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;font-size:small;color:#000000">Is there a way to force extra debugging output in segfault scenarios?</div></div><div class="m_3067247184804260541HOEnZb"><div class="m_3067247184804260541h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 16, 2017 at 1:42 PM, Lukas Larsson <span dir="ltr"><<a href="mailto:lukas@erlang.org" target="_blank">lukas@erlang.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>In order to debug this we need access to a core file generated from the crash and also the beam.smp binary that was used to run it.</div><div><br></div><div>Lukas<br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_3067247184804260541m_7323346486169285344h5">On Wed, Aug 16, 2017 at 1:20 PM, Gerhard Lazu <span dir="ltr"><<a href="mailto:gerhard@lazu.co.uk" target="_blank">gerhard@lazu.co.uk</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_3067247184804260541m_7323346486169285344h5"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">Hi,</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">A RabbitMQ 3.6.11-rc.3 node crashed when OTP 20.0.2 segfaulted. This is the only information that we have about the Erlang VM itself:</div><div style="font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div><div style="color:rgb(0,0,0);font-size:small"><font face="monospace, monospace" style="background-color:rgb(238,238,238)"> Aug 16 04:08:59 localhost kernel: [61139.180699] 2_scheduler[15162]: segfault at 5c ip 00000000005ba760 sp 00007f8fea1ffc40 err</font></div><div style="color:rgb(0,0,0);font-size:small"><font face="monospace, monospace" style="background-color:rgb(238,238,238)">or 4 in beam.smp[400000+333000]</font></div><div style="color:rgb(0,0,0);font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div><div><font color="#000000" face="arial, helvetica, sans-serif">This is the last line logged by RMQ, 2 seconds before the Erlang VM segfault:</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="monospace, monospace" style="background-color:rgb(238,238,238)"> =INFO REPORT==== 16-Aug-2017::04:08:57 === closing AMQP connection <0.24230.9> (<a href="http://10.0.16.35:42562" target="_blank">10.0.16.35:42562</a> -> <a href="http://10.0.16.23:5672" target="_blank">10.0.16.23:5672</a>, vhost: '/', user: 'admin')</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif">beam.smp runs with the following flags:</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="monospace, monospace" style="background-color:rgb(238,238,238)"> /var/vcap/packages/erlang-20.0<wbr>.2/lib/erlang/erts-9.0.2/bin/b<wbr>eam.smp -W w -A 64 -P 1048576 -t 5000000 -stbt db -zdbbl 128000 -K true -- -root /var/vcap/packages/erlang-20.0<wbr>.2/lib/erlang -progname erl -- -home /home/vcap -- -pa /var/vcap/jobs/rabbitmq-server<wbr>/packages/rabbitmq-server/ebin -noshell -noinput -s rabbit boot -sname rabbit@rmq0-rmq-gcp-36 -boot start_sasl -config /var/vcap/jobs/rabbitmq-server<wbr>/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/vcap/sys/log/rabbi<wbr>tmq-server/rabbit@rmq0-rmq-gcp<wbr>-36.log"} -rabbit sasl_error_logger {file,"/var/vcap/sys/log/rabbi<wbr>tmq-server/rabbit@rmq0-rmq-gcp<wbr>-36-sasl.log"} -rabbit enabled_plugins_file "/var/vcap/jobs/rabbitmq-serve<wbr>r/packages/rabbitmq-server/etc<wbr>/rabbitmq/enabled_plugins" -rabbit plugins_dir "/var/vcap/jobs/rabbitmq-serve<wbr>r/packages/rabbitmq-server/plu<wbr>gins" -rabbit plugins_expand_dir "/var/vcap/store/rabbitmq-serv<wbr>er/mnesia/rabbit@rmq0-rmq-gcp-<wbr>36-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/vcap/store/rabbitmq-serv<wbr>er/mnesia/rabbit@rmq0-rmq-gcp-<wbr>36" -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif">We are running on:</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="monospace, monospace" style="background-color:rgb(238,238,238)"> 4.4.0-83-generic #106~14.04.1-Ubuntu SMP Mon Jun 26 18:10:19 UTC 2017 x86_64 GNU/Linux</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif">When this happens again, how do I capture more debugging information? I would like to file an OTP bug report, but I'm not sure that we have enough information.</font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif">Apparently, this was reported at least once before, against OTP 18.1: <a href="https://github.com/rabbitmq/rabbitmq-server/issues/459" target="_blank">https://github.com/rabbi<wbr>tmq/rabbitmq-server/issues/459</a></font></div><div><font color="#000000" face="arial, helvetica, sans-serif"><br></font></div><div><font color="#000000" face="arial, helvetica, sans-serif">Thank you, Gerhard.</font></div></div></div></div>
<br></div></div>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/list<wbr>info/erlang-questions</a><br>
<br></blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/list<wbr>info/erlang-questions</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>