[erlang-questions] Garbage Collection, BEAM memory and Erlang memory

[Valentin Micic]

If you going to copy, you may just as well use lists instead of binary…

V/


On 22 Jan 2015, at 7:38 PM, Roberto Ostinelli wrote:

> Here's something I've tried which is successful in avoiding the memory increase for binaries.
> 
> Inside a loop, I used to have:
> 
>     [...]
>     <<Body:Len/binary, "\r\n", Rest/binary>> = Data,
>     loop(Body, Rest);
> 
> Now I force a binary copy to ensure that the reference to the original full binary is easily removed:
> 
>     [...]
>     <<Body0:Len/binary, "\r\n", Rest0/binary>> = Data,
>     Body = binary:copy(Body0),
>     Rest = binary:copy(Rest0),
>     loop(Body, Rest);
> 
> This seems to have stabilized the memory usage reported by erlang:memory/0.
> 
> However:
> I believe this can only work if the copied binary are heap and not ref-c, is this correct?
> Unfortunately, the BEAM process reported RES memory sill keeps growing.
> Any other ideas?
> r.
> 
> 
> 
> 
> On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli <roberto@REDACTED> wrote:
> Thank you Robert.
> I'm going to try a selective fullsweep_after.
> 
> Could this also justify the process memory increase (which is more significant)?
> 
> 
> 
> On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding <rvirding@REDACTED> wrote:
> One thing you can see is that the size of the binary data is growing. This space contains the large binaries (> 64 bytes) which are sent in messages between processes. While this means that the messages become (much) smaller and faster to send it takes a much longer time to detect that they are no longer alive and can be reclaimed. Basically it takes until all the processes they have passed through does a full garbage collection. Setting fullsweep_after to 0 and doing explicit garbage collects speeds up reclaiming the binaries.
> 
> You could be much more selective in which processes you set fullsweep_after to 0 and which ones you explicitly garbage collect.
> 
> I don't know if the is *the* problem but it is *a* problem you have.
> 
> Robert
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150122/7fbb5c23/attachment.htm>