Garbage collection of binaries

[Sean Hinde]

On Monday, October 20, 2003, at 11:12  am, Erik Stenman wrote:

>
>> Thomas Lindgren Wrote
>> --- Joachim Durchholz <joachim.durchholz@REDACTED> wrote:
>>> An even more modular approach would be to make the
>>> binaries normal
>>> citizens of garbage collected memory. This would
>>> also simplify the
>>> emulator code, I'd think :-)
>>
>> Storing binaries globally means message passing needs
>> not copy binaries. That was the reason I heard, at
>> least.
>
> Binaries would be copyied not onlyby  message passing,
> but by the garbage collector itself.
> ERTS currently uses a generational *copying* GC, hence
> storing the binaries directly on the heap without
> rewriting the GC would mean that the binary would be
> copied at each major GC.
>
> /Erik
>
>

This is great, and non copying of binaries is a fantastic feature which 
moves Erlang into the realms of carrying traffic as well as control, 
but we still need a solution to the problem that a process will merrily 
use up all 4G of beam memory space without doing a garbage collection..

I think that a solution in which a process keeps track of its own large 
binary allocations and kicks of a GC if this gets too big will be a 
good step. It will not cover all cases where tracking reference counts 
across all processes might lead to a different answer, but it would 
cover the common case of a single process processing a large amount of 
binary data and would provide good protection in the other cases.

Sean