<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">If you going to copy, you may just as well use lists instead of binary…<div><br></div><div>V/</div><div><br></div><div><div><div><br><div><div><div>On 22 Jan 2015, at 7:38 PM, Roberto Ostinelli wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">Here's something I've tried which is successful in avoiding the memory increase for binaries.<div><br></div><div>Inside a loop, I used to have:</div><div><br></div><div><font face="monospace, monospace"> [...]</font></div><div><font face="monospace, monospace"> <<Body:Len/binary, "\r\n", Rest/binary>> = Data,<br></font></div><div><div><font face="monospace, monospace"> loop(Body, Rest);</font><br></div></div><div><br></div><div>Now I force a binary copy to ensure that the reference to the original full binary is easily removed:</div><div><br class=""><font face="monospace, monospace"> [...]<br></font></div><div><div><font face="monospace, monospace"> <<Body0:Len/binary, "\r\n", Rest0/binary>> = Data,</font></div><div><font face="monospace, monospace"> Body = binary:copy(Body0),</font></div><div><font face="monospace, monospace"> Rest = binary:copy(Rest0),</font></div><div><font face="monospace, monospace"> loop(Body, Rest);</font><br></div></div><div><br></div><div><div>This seems to have stabilized the memory usage reported by <font face="monospace, monospace">erlang:memory/0</font>.<br></div><div><br></div><div>However:</div><div><ul><li>I believe this can only work if the copied binary are <i>heap</i> and not <i>ref-c</i>, is this correct?</li><li>Unfortunately, the BEAM process reported RES memory sill keeps growing.</li></ul></div><div>Any other ideas?</div><div>r.</div><div><br></div></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli <span dir="ltr"><<a href="mailto:roberto@widetag.com" target="_blank">roberto@widetag.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thank you Robert.<div>I'm going to try a selective fullsweep_after.</div><div><br></div><div>Could this also justify the process memory increase (which is more significant)?<br></div><span class=""><div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding <span dir="ltr"><<a href="mailto:rvirding@gmail.com" target="_blank">rvirding@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>One thing you can see is that the size of the binary data is growing. This space contains the large binaries (> 64 bytes) which are sent in messages between processes. While this means that the messages become (much) smaller and faster to send it takes a much longer time to detect that they are no longer alive and can be reclaimed. Basically it takes until all the processes they have passed through does a full garbage collection. Setting fullsweep_after to 0 and doing explicit garbage collects speeds up reclaiming the binaries.<br><br></div><div>You could be much more selective in which processes you set fullsweep_after to 0 and which ones you explicitly garbage collect.<br><br></div>I don't know if the is *the* problem but it is *a* problem you have.<br><br></div>Robert</div>
</blockquote></div><br></div></div></span></div>
</blockquote></div><br></div>
_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>http://erlang.org/mailman/listinfo/erlang-questions<br></blockquote></div><br></div></div></div></div></body></html>