<div dir="ltr">In our case, a key insight is something Robert mentioned earlier. <div>To paraphrase it, "Your memory is not going to be reclaimed till *every* process that touched it has either been GC'd, or sleeps with the fishes".</div><div>And our code had a few router processes that, quite literally, did nothing but pass binaries on from point A to point B - no reason to worry about *those* right? (hint: wrong)<br><div><br><div>In the short run, walking the process chain and doing full-sweeps on *every* process that might (!!!) touch the binary. (we did that)</div><div>In the longer run, see if you can swap out the routing processes with versions that respawn themselves in some form or the other (we did that too. eventually. was harder)</div></div></div><div><br></div><div>Cheers</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 23, 2015 at 12:14 AM, Dan Gudmundsson <span dir="ltr"><<a href="mailto:dangud@gmail.com" target="_blank">dangud@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Thu, Jan 22, 2015 at 6:38 PM, Roberto Ostinelli <span dir="ltr"><<a href="mailto:roberto@widetag.com" target="_blank">roberto@widetag.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Here's something I've tried which is successful in avoiding the memory increase for binaries.<div><br></div><div>Inside a loop, I used to have:</div><div><br></div><div><font face="monospace, monospace"> [...]</font></div><div><font face="monospace, monospace"> <<Body:Len/binary, "\r\n", Rest/binary>> = Data,<br></font></div><div><div><font face="monospace, monospace"> loop(Body, Rest);</font><br></div></div><div><br></div><div>Now I force a binary copy to ensure that the reference to the original full binary is easily removed:</div><div><br><font face="monospace, monospace"> [...]<br></font></div><div><div><font face="monospace, monospace"> <<Body0:Len/binary, "\r\n", Rest0/binary>> = Data,</font></div><div><font face="monospace, monospace"> Body = binary:copy(Body0),</font></div><div><font face="monospace, monospace"> Rest = binary:copy(Rest0),</font></div><div><font face="monospace, monospace"> loop(Body, Rest);</font><br></div></div><div><br></div><div><div>This seems to have stabilized the memory usage reported by <font face="monospace, monospace">erlang:memory/0</font>.<br></div><div><br></div><div>However:</div><div><ul><li>I believe this can only work if the copied binary are <i>heap</i> and not <i>ref-c</i>, is this correct?</li></ul></div></div></div></blockquote></span><div>Binary copy makes a new copy of the binary regardless of size, but it is only useful on ref-c binaries,</div><div>and should be used to avoid keeping references to large binaries, when you keep and use a small part </div><div>of the original binary. But in this case you make a copy of Data (-2 bytes) and thus will double up the memory until Data is gc'ed and refc reaches zero. So I do not see the point of the above nor of making it a list which will</div><div>only explode your memory even more.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><div dir="ltr"><div><div><ul><li>Unfortunately, the BEAM process reported RES memory sill keeps growing.</li></ul></div><div>Any other ideas?</div><span><font color="#888888"><div>r.</div><div><br></div></font></span></div><div><br></div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli <span dir="ltr"><<a href="mailto:roberto@widetag.com" target="_blank">roberto@widetag.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thank you Robert.<div>I'm going to try a selective fullsweep_after.</div><div><br></div><div>Could this also justify the process memory increase (which is more significant)?<br></div><span><div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding <span dir="ltr"><<a href="mailto:rvirding@gmail.com" target="_blank">rvirding@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>One thing you can see is that the size of the binary data is growing. This space contains the large binaries (> 64 bytes) which are sent in messages between processes. While this means that the messages become (much) smaller and faster to send it takes a much longer time to detect that they are no longer alive and can be reclaimed. Basically it takes until all the processes they have passed through does a full garbage collection. Setting fullsweep_after to 0 and doing explicit garbage collects speeds up reclaiming the binaries.<br><br></div><div>You could be much more selective in which processes you set fullsweep_after to 0 and which ones you explicitly garbage collect.<br><br></div>I don't know if the is *the* problem but it is *a* problem you have.<br><br></div>Robert</div>
</blockquote></div><br></div></div></span></div>
</blockquote></div><br></div>
</div></div><br></span><span class="">_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></span></blockquote></div><br></div></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div style="font-family:Helvetica;word-wrap:break-word"><div style="margin:0in 0in 0.0001pt"><font color="#1f497d" face="Calibri, sans-serif"><span style="font-size:15px"><b><div style="font-style:italic;margin:0px;font-family:Calibri"><b style="color:rgb(17,85,204)"><a href="http://www.gravatar.com/avatar/204a87f81a0d9764c1f3364f53e8facf.png" style="color:rgb(17,85,204)" target="_blank">Mahesh Paolini-Subramanya</a></b></div><div style="margin:0px;font-family:Calibri"><span style="font-weight:normal">That tall bald Indian guy..</span><br></div></b></span></font></div></div><div style="font-family:Helvetica;word-wrap:break-word"><div style="margin:0in 0in 0.0001pt"><font color="#1f497d" face="Calibri, sans-serif"><span style="font-size:15px"><b><div style="margin:0px;font-family:Calibri"><div style="font-family:Helvetica;word-wrap:break-word"><div style="margin:0in 0in 0.0001pt;font-size:12pt;font-family:'Times New Roman',serif"><span style="font-size:11pt;font-family:Calibri,sans-serif"><div style="margin:0px;font-family:Calibri;color:rgb(1,108,226)"><a href="https://plus.google.com/u/0/108074935470209044442/posts" style="color:rgb(17,85,204)" target="_blank">Google+</a><span style="color:rgb(31,73,125)"> | <a href="http://dieswaytoofast.blogspot.com/" style="color:rgb(17,85,204)" target="_blank"><span style="color:rgb(1,108,226)">Blog</span></a></span> <span style="color:rgb(31,73,125)"> | <span style="color:rgb(1,108,226)"><a href="https://twitter.com/dieswaytoofast" style="color:rgb(17,85,204)" target="_blank">Twitter</a></span></span><span style="color:rgb(31,73,125)"> | </span><a href="http://www.linkedin.com/in/dieswaytoofast" style="color:rgb(17,85,204)" target="_blank">LinkedIn</a></div></span></div></div></div></b></span></font></div></div></div></div>
</div>