<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, May 3, 2018 at 12:18 PM, Gerhard Lazu <span dir="ltr"><<a href="mailto:gerhard@lazu.co.uk" target="_blank">gerhard@lazu.co.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Lukas,<div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>Why do you not use erlang:memory() as the base for whether you can accept more messages? Having a low memory utilisation is not bad in itself, unless of course some other program on the same machine needs the memory.</div></div></div></div></blockquote><div><br></div></span><div>We used to use <font face="monospace, monospace" style="background-color:rgb(238,238,238)">erlang:memory()</font>, but we've learned that it doesn't work well in practice [1]. Linux OOM will take action based on RSS, not Erlang allocated memory.</div></div></div></div></blockquote><div><br></div><div>Yes, looking at erlang:memory() could make you end up in those scenarios. However it should be possible to look at recon_alloc:memory(unused), to get a ballpark figure about how much memory that the Erlang VM has reserved that it is not using at the moment.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>Looking at the used memory under load and after, at peak the allocated memory is 1510 MB and then after it is 577 MB. So about 2/3rds of the allocated memory was returned to the OS. While this is not perfect, it is not terrible either. Reducing it further may not be easy.<br></div></div></div></div></blockquote><div><br></div></span><div>Your observation is true and accurate. It's also true and accurate that out of 577MB allocated, 300MB is used & 277MB is unused, meaning that almost half of the allocated memory is not used.</div><div><br></div><div>I understand that it may not be easy to reduce the unused memory, but all I'm thinking is that whilst this unused memory might seem small in this particular scenario, what happens when the Erlang VM has 60GB allocated?</div></div></div></div></blockquote><div><br></div><div>One thing that you could try is to see if malloc does a better job than erts does. In general the erts allocators are better at scalability, while malloc is better at performance. I don't know which is better at dealing with fragmentation, use "+MBe false" to disable it for binary alloc. The really bad part about this is that you loose all statistics. So if you do get into other issues it will be much harder to figure out what is going on.</div><div><br></div><div>You could also try to play with the sbct to see if you get better allocations using a lower value. This will cause more allocations to be places into sbcs which could be good for fragmentation. You seem to have an average block size of about 2 kb, so I would try setting the sbct to about double that to start with and see if you notice any difference. It's hard to know what be a good value with the OTP-20 instrumentation, so you will have to try and see if you get any difference.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>Would it help if we can show the impact of this behaviour on hosts with larger memory usage?<br></div></div></div></div></blockquote><div><br></div><div>No I don't think so. The behavior should be similar just with scaled up values. I suppose that it would be a good idea to verify that if you can.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><span class="gmail-"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span><div>Can you reproduce the behaviour? Would it be possible to get a recon_alloc snaphot during and after load?<br></div></span></div></div></div></blockquote><div><br></div></span><div>I'm sharing <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">recon_alloc snapshots </span>during & after load for the following:</div><div><br></div><div>1. erts_alloc defaults (<font face="monospace, monospace" style="background-color:rgb(238,238,238)">lmbcs 5120</font>) [2]</div><div>2. <span style="background-color:rgb(238,238,238)">-</span><font face="monospace, monospace" style="background-color:rgb(238,238,238)">MBlmbcs 512</font> [3]</div><div> </div><div>I've also captured a during & after load screenshots of the 2 configurations running side-by-side (left is erts_alloc defaults (<font face="monospace, monospace" style="background-color:rgb(238,238,238)">lmbcs 5120</font>), right is <span style="color:rgb(34,34,34);font-family:monospace,monospace;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(238,238,238);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">-MBlmbcs 512</span>) [4].</div><div><br></div><div>While our initial configuration used a few more flags, <span style="color:rgb(80,0,80);font-family:monospace,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(238,238,238);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">-MHas ageffcbf -MBas ageffcbf -MHlmbcs 512 -MBlmbcs 512</span>, I've kept things as simple as possible on this run, and only used <font face="monospace, monospace" style="background-color:rgb(238,238,238)">-MBlmbcs 512</font></div><span class="gmail-"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span><div></div></span><div>Have you seen <a href="https://github.com/erlang/otp/pull/1790" target="_blank">https://github.com/erlang<wbr>/otp/pull/1790</a> that was just merged to master with the accompanying blog post: <a href="http://blog.erlang.org/Memory-instrumentation-in-OTP-21/" target="_blank">http://blog.erlang.org/M<wbr>emory-instrumentation-in-OTP-2<wbr>1/</a>?<br></div></div></div></div></blockquote><div><br></div></span><div>I haven't, thank you for sharing. We are waiting on Elixir #6611 before we can test against OTP 21.0-rc1 [5].</div><span class="gmail-"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>I assume that you have tried ageffcaobf?</div></div></div></div></blockquote><div><br></div></span><div>Yes, we have tried all allocation strategies. <font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcbf</font> resulted in "spikier" CPU and dirty mem writeback, but also sharper drops in dirty mem writeback. Under load, <font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcbf</font> had 1% lower RSS usage, and 2.5% lower unused memory than <span style="color:rgb(34,34,34);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline;background-color:rgb(238,238,238)"><font face="monospace, monospace">ageffcaobf</font></span>. After load however, <font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcbf</font> had 5% lower RSS usage & 4% lower unused memory. In conclusion, <font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcbf</font> proved the best out of all allocation strategies.</div><div> <br></div></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>Here is a side-by-side comparison of <span style="background-color:rgb(238,238,238)"><font face="monospace, monospace">-MBas </font></span><font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcaobf -MBlmbcs 512</font> (left) vs <font face="monospace, monospace" style="background-color:rgb(238,238,238)">-MBas </font><font face="monospace, monospace" style="background-color:rgb(238,238,238)">ageffcbf -MBlmbcs 512</font> (right) [6], and the relevant recon_alloc snapshots [7].</div></div></div></div></blockquote><div><br></div><div><div>While aobf should be a little bit better, I suspect that since we introduced the carrier pool and thus use the the carrier strategies the difference is within the error margins, especially when you run a system with lots of small carriers.</div></div><div><br></div><div>On a related point, this discussion has caused me to start looking at using madvise/VirtualAlloc to let the OS know that pages within carriers are not used by erts any more. I'm not sure how that will interact with RSS as from I I've been able to figure out the pages remain associated with the program until the OS needs them so satisfy some memory request. Any such feature won't be in OTP-21, but may be added later.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>Thank you Lukas for helping out with this, Gerhard.</div><div><br></div><div>[1] <a href="https://github.com/rabbitmq/rabbitmq-server/issues/1223" target="_blank">https://github.com/rabbitm<wbr>q/rabbitmq-server/issues/1223</a> & <a href="https://github.com/rabbitmq/rabbitmq-server/pull/1259#issuecomment-308428057" target="_blank">https://github.com/rabbitmq/<wbr>rabbitmq-server/pull/1259#issu<wbr>ecomment-308428057</a> - the entire PR context is valuable and relevant</div><div>[2] <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBlmbcs_5120.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazo<wbr>naws.com/rabbitmq-share/memory<wbr>-allocators/during-MBlmbcs_<wbr>5120.recon_alloc.snapshot</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBlmbcs_5120.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBlmbcs_5120.<wbr>recon_alloc.snapshot</a></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[3] <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBlmbcs_512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazo<wbr>naws.com/rabbitmq-share/memory<wbr>-allocators/during-MBlmbcs_<wbr>512.recon_alloc.snapshot</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBlmbcs_512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBlmbcs_512.<wbr>recon_alloc.snapshot</a></span></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[4] <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBlmbcs_5120-vs-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazo<wbr>naws.com/rabbitmq-share/memory<wbr>-allocators/during-MBlmbcs_<wbr>5120-vs-MBlmbcs_512.png</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBlmbcs_5120-vs-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBlmbcs_5120-<wbr>vs-MBlmbcs_512.png</a></span></span></span></span></div><div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[5] </span><font color="#1155cc"><u><a href="https://github.com/elixir-lang/elixir/issues/6611#issuecomment-386208496" target="_blank">https://github.com/elixir-<wbr>lang/elixir/issues/6611#issuec<wbr>omment-386208496</a></u></font><br></span></span></span></span></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[6] <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBas_ageffcaobf-MBlmbcs_512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazo<wbr>naws.com/rabbitmq-share/memory<wbr>-allocators/during-MBas_ageffc<wbr>aobf-MBlmbcs_512.recon_alloc.<wbr>snapshot</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBas_ageffcaobf-MBlmbcs_512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBas_ageffcao<wbr>bf-MBlmbcs_512.recon_alloc.<wbr>snapshot</a> + <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBas_ageffcbf-MBlmbcs-512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/during-MBas_ageffcb<wbr>f-MBlmbcs-512.recon_alloc.<wbr>snapshot</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBas_ageffcbf-MBlmbcs-512.recon_alloc.snapshot" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBas_ageffcbf<wbr>-MBlmbcs-512.recon_alloc.<wbr>snapshot</a></span></span></span></span></span></span></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">[7] <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-MBas_ageffcaobf-MBlmbcs_512-vs-MBas_ageffcbf-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazo<wbr>naws.com/rabbitmq-share/memory<wbr>-allocators/during-MBas_ageffc<wbr>aobf-MBlmbcs_512-vs-MBas_<wbr>ageffcbf-MBlmbcs_512.png</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/after-MBas_ageffcaobf-MBlmbcs_512-vs-MBas_ageffcbf-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/after-MBas_ageffcao<wbr>bf-MBlmbcs_512-vs-MBas_<wbr>ageffcbf-MBlmbcs_512.png</a> + <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-cpu-MBas_ageffcaobf-MBlmbcs_512-vs-MBas_ageffcbf-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazona<wbr>ws.com/rabbitmq-share/memory-<wbr>allocators/during-cpu-MBas_<wbr>ageffcaobf-MBlmbcs_512-vs-<wbr>MBas_ageffcbf-MBlmbcs_512.png</a> & <a href="https://s3-eu-west-1.amazonaws.com/rabbitmq-share/memory-allocators/during-memory-MBas_ageffcaobf-MBlmbcs_512-vs-MBas_ageffcbf-MBlmbcs_512.png" target="_blank">https://s3-eu-west-1.amazonaws<wbr>.com/rabbitmq-share/memory-<wbr>allocators/during-memory-MBas_<wbr>ageffcaobf-MBlmbcs_512-vs-<wbr>MBas_ageffcbf-MBlmbcs_512.png</a></span></span></span></span></span></span></span></div><div><br></div></div></div></div>
<br>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div></div>