<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 9, 2018 at 11:58 AM, pablo platt <span dir="ltr"><<a href="mailto:pablo.platt@gmail.com" target="_blank">pablo.platt@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-">On Fri, Mar 9, 2018 at 12:38 PM, Lukas Larsson <span dir="ltr"><<a href="mailto:lukas@erlang.org" target="_blank">lukas@erlang.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello!<br><div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-m_3949143316883808096gmail-">On Thu, Mar 8, 2018 at 8:43 PM, pablo platt <span dir="ltr"><<a href="mailto:pablo.platt@gmail.com" target="_blank">pablo.platt@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>Hello,<br><br></div><div>Updating with my test.<br><br></div>I've been using master in production for two days on 4 servers.<br></div>Each server has 4 vCPUs.<br>Each server handles about 10K UDP packets per second. Approx 1K incoming packets per second and 9K outgoing packets per second.<br><br></div>I didn't see any issues. Seems to work fine.<br></div></div></div></div></blockquote><div> </div></span><div>That's great! Thanks for testing it! </div><span class="gmail-m_3949143316883808096gmail-"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div></div>CPU load in OTP Master 21 increased compared to OTP 20 from 58% to 68%. Does this make sense?<br></div></div></div></blockquote><div><br></div></span><div>hmm, no not really. I would have expected it to decrease.</div><div><br></div><div>Would you mind helping me to figure out why the CPU usage has gone up?</div><div><br></div><div>To start with I'd like to have a look at the output of</div><div><br></div><div>1> msacc:start(30000), msacc:print().</div><div><br></div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>for both versions in the Erlang shell. Using this I hope to be able to narrow down where the extra CPU time is being spent.</div><div><br></div><div>Also it would great if you could run a few perf commands to compare the systems.</div><div><br></div><div>Ideally it would be be best if you could recompile beam with the --build-id linker flag. i.e.</div><div><br></div><div>> ./configure LDLAGS="-Wl,--build-id"</div></div></div></div></blockquote></span><div><br>I'll try to compile with kerl:<br>export LDLAGS="-Wl,--build-id" && kerl build git <a href="https://github.com/erlang/otp" target="_blank">https://github.com/erlang/otp</a> 9bc4a096025254aed157e4753743be<wbr>61ce1f7489 master<br><br></div><div>How can I verify that the flag was actually used?<br></div></div></div></div></blockquote><div><br></div><div>If you do "file path/to/beam.smp" you should get something like:</div><div><br></div><div>> file bin/x86_64-unknown-linux-gnu/beam.smp<br></div><div>bin/x86_64-unknown-linux-gnu/beam.smp: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=1a8ff129828d9ed4a8197d47e7731b10015b2456, not stripped<br></div><div><br></div><div>notice the BuildID[sha1] thing.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span class="gmail-"><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>That way it is possible to use "perf archive"[1] to collect the symbols of beam.smp.</div><div><br></div><div>> sudo perf stat -d -p $BEAMPID -- sleep 30 2> stat.log</div><div>> sudo perf record -g -p $BEAMPID -- sleep 30</div><div>> sudo perf archive</div><div>> sudo tar czf $BEAMPID.tar.gz stat.log perf.data.tar.bz2 perf.data</div><div><br></div></div></div></div></blockquote><div><br></div></span><div>Can I do it on a dev machine with a smaller load or does it have to be on a real production server?<br></div></div></div></div></blockquote><div><br></div><div>If you can observe the same difference in a small system then that is fine.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>What's the effect of perf (and msacc) on a production system? When I tried to do erlang profiling in the past it crashed my server.<br></div></div></div></div></blockquote><div><br></div><div>It should be very minimal. I double that you will notice it. If you are unsure, you can change the frequency that perf collects data by adding the "-F NUMBER" flag to "perf report" with a lower than default frequency. The default is 4000.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> <br></div><span class="gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>and send me the tar.gz for OTP-20 and master.</div><div><br></div><div>If you cannot re-compile erts with --build-id, then just do "sudo perf report" and send me a screenshot of the tools shows you.</div><div><br></div><div>If you prefer to this off-list, we can do that as well.</div><div><br></div><div>Lukas</div><div><br></div><div>[1]: If the "perf archive" command does not work (which it doesn't on my machine), just download the script from here: <a href="https://elixir.bootlin.com/linux/v3.18/source/tools/perf/perf-archive.sh" target="_blank">https://elixir.bootlin.c<wbr>om/linux/v3.18/source/tools/pe<wbr>rf/perf-archive.sh</a></div></div></div></div>
</blockquote></span></div><br></div></div>
<br>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div></div>