<div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote">On Mon, Jun 18, 2018 at 3:57 PM, Lukas Larsson <span dir="ltr"><<a href="mailto:lukas@erlang.org" target="_blank">lukas@erlang.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_quote"><span class="m_412083880446826936gmail-m_-1970298089873718126m_-2319506004081891027gmail-m_-8427482326205488689m_1730979584661245648gmail-m_-6991735491702054575gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br></blockquote><div>Something like <a href="https://github.com/falkevik/otp/commit/444fb00ff2a9d1f40a8c66f48bea1cf3f07ca86c" target="_blank">https://github.com/falkevik/ot<wbr>p/commit/444fb00ff2a9d1f40a8c6<wbr>6f48bea1cf3f07ca86c</a> ?</div></div></div></div></blockquote><div><br></div></span><div>I was thinking something more like this: <a href="https://github.com/garazdawi/otp/tree/lukas/erts/realloc_thr_pref_shrink_th" target="_blank">https://github.com/garazdawi/o<wbr>tp/tree/lukas/erts/realloc_thr<wbr>_pref_shrink_th</a></div><div><br></div><div>This makes it so that the absolute and relative single block shrink thesholds are respected for reallocs made on a remote scheduler. This should solve the problem that you have found also, through as I still haven't reproduced it I can't test that it actually solves it.</div><span class="m_412083880446826936gmail-m_-1970298089873718126m_-2319506004081891027gmail-m_-8427482326205488689m_1730979584661245648gmail-m_-6991735491702054575gmail-HOEnZb"><font color="#888888"><div><br></div></font></span></div></div></blockquote><div> </div><div>Are you adding the multicast network to the loop back interface? Using some other interface that does not allow multicast traffic?</div></div><br></div><div class="gmail_extra">I have been able to reproduce it in macOS Sierra (Darwin Kernel Version 16.7.0: Fri Apr 27 17:59:46 PDT 2018; root:xnu-3789.73.13~1/RELEASE_<wbr>X86_64 x86_64)</div><div class="gmail_extra">and also on Ubuntu Linux 16.04 LTS with kernel 4.4.0-112-generic.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Danil Zagoskin, do you have time to try and see if there is any change in the behaviour for you with a patched system?<br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra">With the bugfix, the beam is spending most of its time in sched_spin_wait on linux and sched_yield on macOS.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Stats below are from OTP 21 and then OTP21 with bugfix on linux</div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra"><font face="monospace, monospace">Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Eshell V10.0  (abort with ^G)</font></div><div class="gmail_extra"><font face="monospace, monospace">1> udptest:start_sender({239,9,9,<wbr>9}, 3999).</font></div><div class="gmail_extra"><font face="monospace, monospace"><0.78.0></font></div><div class="gmail_extra"><font face="monospace, monospace">2> [udptest:start_reader({239,9,9<wbr>,9}, 3999) || _ <- lists:seq(1, 40)].</font></div><div class="gmail_extra"><font face="monospace, monospace">[<0.80.0>,<0.81.0>,<0.82.0>,<0<wbr>.83.0>,<0.84.0>,<0.85.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.86.0>,<0.87.0>,<0.88.0>,<0<wbr>.89.0>,<0.90.0>,<0.91.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.92.0>,<0.93.0>,<0.94.0>,<0<wbr>.95.0>,<0.96.0>,<0.97.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.98.0>,<0.99.0>,<0.100.0>,<<wbr>0.101.0>,<0.102.0>,<0.103.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.104.0>,<0.105.0>,<0.106.0><wbr>,<0.107.0>,<0.108.0>|...]</font></div><div class="gmail_extra"><font face="monospace, monospace">3> msacc:start(10000), msacc:print().</font></div><div class="gmail_extra"><font face="monospace, monospace">Average thread real-time    : 10002138 us</font></div><div class="gmail_extra"><font face="monospace, monospace">Accumulated system run-time : 62576106 us</font></div><div class="gmail_extra"><font face="monospace, monospace">Average scheduler run-time  :  7775309 us</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">        Thread      aux check_io emulator       gc    other     port    sleep</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Stats per thread:</font></div><div class="gmail_extra"><font face="monospace, monospace">     async( 0)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">       aux( 1)    0.06%    0.96%    0.00%    0.00%    0.07%    0.00%   98.90%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 1)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 2)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 3)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 4)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 5)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 6)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 7)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 8)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 1)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 2)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 3)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 4)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 5)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 6)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 7)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 8)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 9)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s(10)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">      poll( 0)    0.00%    2.63%    0.00%    0.00%    0.00%    0.00%   97.37%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 1)    0.82%    0.00%    0.52%    0.35%    7.33%   67.91%   23.08%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 2)    0.76%    0.00%    0.79%    0.34%    7.30%   68.61%   22.21%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 3)    0.74%    0.00%    0.83%    0.33%    7.17%   68.54%   22.38%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 4)    0.76%    0.00%    0.72%    0.34%    7.38%   68.84%   21.96%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 5)    0.79%    0.00%    0.82%    0.34%    7.21%   68.94%   21.90%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 6)    0.80%    0.00%    0.74%    0.31%    7.15%   68.56%   22.44%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 7)    0.87%    0.00%    0.72%    0.37%    7.24%   68.43%   22.36%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 8)    0.77%    0.00%    0.68%    0.35%    7.41%   69.02%   21.77%</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Stats per type:</font></div><div class="gmail_extra"><font face="monospace, monospace">         async    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">           aux    0.06%    0.96%    0.00%    0.00%    0.07%    0.00%   98.90%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_sche    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_sched    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">          poll    0.00%    2.63%    0.00%    0.00%    0.00%    0.00%   97.37%</font></div><div class="gmail_extra"><font face="monospace, monospace">     scheduler    0.79%    0.00%    0.73%    0.34%    7.27%   68.61%   22.26%</font></div><div class="gmail_extra"><font face="monospace, monospace">ok</font></div><div class="gmail_extra"><font face="monospace, monospace">4> </font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">$ ../src/udp_performace_bug_fix/<wbr>bin/erl</font></div><div class="gmail_extra"><font face="monospace, monospace">Erlang/OTP 21 [RELEASE CANDIDATE 2] [erts-9.3.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Eshell V9.3.2  (abort with ^G)</font></div><div class="gmail_extra"><font face="monospace, monospace">1> udptest:start_sender({239,9,9,<wbr>9}, 3999).</font></div><div class="gmail_extra"><font face="monospace, monospace"><0.78.0></font></div><div class="gmail_extra"><font face="monospace, monospace">2> [udptest:start_reader({239,9,9<wbr>,9}, 3999) || _ <- lists:seq(1, 40)].</font></div><div class="gmail_extra"><font face="monospace, monospace">[<0.80.0>,<0.81.0>,<0.82.0>,<0<wbr>.83.0>,<0.84.0>,<0.85.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.86.0>,<0.87.0>,<0.88.0>,<0<wbr>.89.0>,<0.90.0>,<0.91.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.92.0>,<0.93.0>,<0.94.0>,<0<wbr>.95.0>,<0.96.0>,<0.97.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.98.0>,<0.99.0>,<0.100.0>,<<wbr>0.101.0>,<0.102.0>,<0.103.0>,</font></div><div class="gmail_extra"><font face="monospace, monospace"> <0.104.0>,<0.105.0>,<0.106.0><wbr>,<0.107.0>,<0.108.0>|...]</font></div><div class="gmail_extra"><font face="monospace, monospace">3> msacc:start(10000), msacc:print().</font></div><div class="gmail_extra"><font face="monospace, monospace">Average thread real-time    : 10000416 us</font></div><div class="gmail_extra"><font face="monospace, monospace">Accumulated system run-time : 19541445 us</font></div><div class="gmail_extra"><font face="monospace, monospace">Average scheduler run-time  :  2279822 us</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">        Thread      aux check_io emulator       gc    other     port    sleep</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Stats per thread:</font></div><div class="gmail_extra"><font face="monospace, monospace">     async( 0)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">       aux( 1)    0.00%   10.72%    0.00%    0.00%    0.00%    0.00%   89.28%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 1)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 2)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 3)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 4)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 5)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 6)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 7)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_( 8)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 1)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 2)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 3)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 4)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 5)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 6)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 7)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 8)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s( 9)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_s(10)    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">      poll( 0)    0.00%    2.31%    0.00%    0.00%    0.00%    0.00%   97.69%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 1)    0.46%    0.00%    0.43%    0.14%   17.57%    4.07%   77.33%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 2)    0.50%    0.00%    0.44%    0.15%   17.13%    4.21%   77.57%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 3)    0.49%    0.00%    0.47%    0.16%   17.07%    4.35%   77.46%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 4)    0.45%    0.00%    0.42%    0.14%   16.25%    4.02%   78.72%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 5)    0.48%    0.00%    0.42%    0.14%   17.04%    4.04%   77.88%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 6)    0.46%    0.00%    0.39%    0.14%   16.57%    3.77%   78.67%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 7)    0.53%    0.00%    1.88%    0.27%   17.60%    8.22%   71.50%</font></div><div class="gmail_extra"><font face="monospace, monospace"> scheduler( 8)    0.47%    0.00%    0.42%    0.15%   16.60%    3.86%   78.50%</font></div><div class="gmail_extra"><font face="monospace, monospace"><br></font></div><div class="gmail_extra"><font face="monospace, monospace">Stats per type:</font></div><div class="gmail_extra"><font face="monospace, monospace">         async    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">           aux    0.00%   10.72%    0.00%    0.00%    0.00%    0.00%   89.28%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_cpu_sche    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">dirty_io_sched    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%</font></div><div class="gmail_extra"><font face="monospace, monospace">          poll    0.00%    2.31%    0.00%    0.00%    0.00%    0.00%   97.69%</font></div><div class="gmail_extra"><font face="monospace, monospace">     scheduler    0.48%    0.00%    0.61%    0.16%   16.98%    4.57%   77.20%</font></div><div class="gmail_extra"><font face="monospace, monospace">ok</font></div><div><br></div><div>Jonas</div></div><div class="gmail_extra"><br></div></div>