[erlang-questions] Garbage collecting binaries

[Ulf Wiger]

f(A) only unbinds the variable in the shell.

The shell still has a history of commands and their results, so you 
would need to ensure that this is not taking up space as well.

BR,
Ulf W

On 7 Oct 2010, at 17:07, tom kelly wrote:

> Hi Nicholas,
> 
> This was just run in the erlang shell, A was a 100000 byte binary, "f(A)."
> told the shell to forget it, ie. set the reference count to zero. There were
> no other references to A in this case nor were other binaries created from
> it. It was gc'd after a while, presumably after 65535 reductions of the
> process that stored it (if that makes sense). My question is how can I
> reduce the fullsweep_after or explicitly force a gc on the heap where it is
> stored.
> 
> //Tom.
> 
> 
> 
> On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette <zeno490@REDACTED>wrote:
> 
>> A bit hard to understand what is happening without seeing f(A)'s code.
>> AFAIK, even if you don't keep a reference to a binary, if you keep a
>> reference to a binary created from it, it'll keep it in full in memory.
>> ie:
>> A = << some large binary>>.
>> <<B:32/binary, _/binary>> = A.
>> If you keep no references to A but you do to B, A will remain in memory
>> because B will have been created as a pointer offset from A (as an
>> optimization).
>> 
>> Nicholas
>> 
>> 
>> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly <ttom.kelly@REDACTED> wrote:
>> 
>>> Hello List,
>>> 
>>> I need some help on garbage collecting binaries.
>>> I have an application that handles large binaries and under load it eats
>>> up
>>> all the available memory then falls over, even when I start all the
>>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}".
>>> 
>>> Reading point 5.15 on the page:
>>> http://www.erlang.org/faq/how_do_i.html
>>> leads me to think that calling the garbage collector on the process that
>>> created a binary will clean it up but my shell experiment below shows me
>>> I'm
>>> wrong. It's now 10 minutes since I've done this experiment and the large
>>> binary is still in memory.
>>> 
>>> Maybe I have to call the garbage collector of the process whose heap is
>>> storing the binary? If so, which process is it?
>>> 
>>> Any pointers as to what I'm doing wrong or mis-understand will be greatly
>>> appreciated.
>>> 
>>> I'm on a legacy R12B5 system.
>>> 
>>> //Tom.
>>> 
>>> 
>>> 
>>> 18> process_info(self(),total_heap_size).
>>> {total_heap_size,4181}
>>> 
>>> 19> A = L(100000).
>>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118,
>>> 180,121,181,160,124,244,140,169,215,31,82,43|...]
>>> 
>>> 20> process_info(self(),total_heap_size).
>>> {total_heap_size,1149851}
>>> 
>>> 21> f(A).
>>> ok
>>> 
>>> 22> erlang:garbage_collect().
>>> true
>>> 
>>> 23> process_info(self(),total_heap_size).
>>> {total_heap_size,3194}
>>> 
>>> 
>>> 24> memory(binary).
>>> 6536
>>> 
>>> 25> A = B(100000).
>>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181,
>>> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>>
>>> 
>>> 26> memory(binary).
>>> 106704
>>> 
>>> 27> f(A).
>>> ok
>>> 
>>> 28> memory(binary).
>>> 106704
>>> 
>>> 29> erlang:garbage_collect().
>>> true
>>> 
>>> 30> memory(binary).
>>> 106560
>>> 
>> 
>> 

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com