<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">If this is scheduler collapse, it would
mean you have a port driver or NIF that has internal latency
greater than 1 millisecond. To handle scheduler collapse, you can
use the erl command line option "-heart" combined with:<br>
heart:set_options([check_schedulers]).<br>
<br>
(see <a class="moz-txt-link-freetext" href="http://erlang.org/doc/man/heart.html#set_options-1">http://erlang.org/doc/man/heart.html#set_options-1</a> )<br>
<br>
That will allow the system to restart when schedulers have
collapsed.<br>
<br>
A test that is meant to cause scheduler collapse is at:<br>
<a class="moz-txt-link-freetext" href="https://github.com/basho/nifwait">https://github.com/basho/nifwait</a><br>
<br>
So you could use that to prove to yourself that the behavior is
the same with the system you are having problems with.<br>
<br>
<br>
On 12/09/2016 08:35 AM, Matthew Evans wrote:<br>
</div>
<blockquote
cite="mid:MWHPR08MB260875E91FEB3B40C5F4BF7A9C870@MWHPR08MB2608.namprd08.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;"
dir="ltr">
<p>Happened again, it appears that code_server is wedged:</p>
<p><br>
</p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">admin@plexxi:~$
doErlangFun "erlang:process_info(whereis(code_server))."</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">[{registered_name,code_server},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {current_function,{code_server,cpc_recv,4}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {initial_call,{erlang,apply,2}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {status,waiting},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {message_queue_len,23},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {messages,[{code_call,<6805.4097.0>,{ensure_loaded,switch_type_module}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.4146.0>,{ensure_loaded,switch_type_module}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.941.0>,{ensure_loaded,pc_port_autoneg}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.541.0>,{ensure_loaded,plexxiStatistics_types}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.520.0>,{ensure_loaded,switch_type_module}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.5123.0>,{ensure_loaded,secondary_erlang_node}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.5122.0>,{ensure_loaded,secondary_erlang_node}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.5162.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.5321.0>,</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{ensure_loaded,mac_entries_record_handler}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.5483.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.6647.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.7232.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.7274.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.7304.0>,{ensure_loaded,icmp}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.8889.0>,</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{ensure_loaded,mac_entries_record_handler}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.8951.0>,</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{ensure_loaded,mac_entries_record_handler}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.576.0>,</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{ensure_loaded,cross_connect_unicast_utils}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.19300.12>,{ensure_loaded,shell}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.20313.12>,{ensure_loaded,shell}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.21339.12>,{ensure_loaded,dbg}},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.31109.13>,get_mode},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{code_call,<6805.1255.14>,get_mode},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{system,{<6805.2521.14>,#Ref<6805.0.23.35356>},get_status}]},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {links,[<6805.11.0>]},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {dictionary,[{any_native_code_loaded,false}]},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {trap_exit,true},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {error_handler,error_handler},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {priority,normal},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {group_leader,<6805.9.0>},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {total_heap_size,86071},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {heap_size,10958},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {stack_size,25},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {reductions,13172282},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {garbage_collection,[{min_bin_vheap_size,46422},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{min_heap_size,233},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{fullsweep_after,65535},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">
{minor_gcs,71}]},</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;"> {suspending,[]}]</span></p>
<p style="margin: 0px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: normal; font-stretch: normal; font-size: 11px;
line-height: normal; font-family: Menlo;">
<span style="font-variant-ligatures: no-common-ligatures;">admin@plexxi:~$ </span></p>
<br>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions-bounces@erlang.org">erlang-questions-bounces@erlang.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:erlang-questions-bounces@erlang.org"><erlang-questions-bounces@erlang.org></a> on behalf of
Matthew Evans <a class="moz-txt-link-rfc2396E" href="mailto:mattevans123@hotmail.com"><mattevans123@hotmail.com></a><br>
<b>Sent:</b> Friday, December 9, 2016 9:56 AM<br>
<b>To:</b> Erlang/OTP discussions<br>
<b>Subject:</b> [erlang-questions] R17 - Possible wedged
scheduler</font>
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper" dir="ltr"
style="font-size:12pt; color:#000000;
font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>We just hit a situation where it appeared that 1
scheduler was wedged. Some parts of our application were
working, but others appeared to be stuck. I could
connect via a cnode application and an escript, but I
couldn't connect via the Erlang shell. We have an
escript that does rpc calls, some worked, others (e.g.
anything to the code server or tracing failed) failed. </p>
<p><br>
</p>
<p>CPU load was minimal at the time, and heart didn't
complain. We only have a single NIF, but this is not
called on this hardware variant. We do use CNODE to talk
to C applications.</p>
<p><br>
</p>
<p>We are running R17, Intel quad core CPU on Debian.</p>
<p><br>
</p>
<p>This is the first time this has been seen, so the
questions are:</p>
<p><br>
</p>
<p>1. Has anyone seen this before?</p>
<p>2. What can we do if we hit this condition in the
future to debug?</p>
<p>3. Since heart doesn't detect this can anyone think of
any alternative mechanisms?</p>
<p><br>
</p>
<p>Thanks</p>
<p><br>
</p>
<p>Matt</p>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
<br>
</body>
</html>