[erlang-questions] Garbage collecting binaries

[tom kelly]

Hi Nicholas,

This was just run in the erlang shell, A was a 100000 byte binary, "f(A)."
told the shell to forget it, ie. set the reference count to zero. There were
no other references to A in this case nor were other binaries created from
it. It was gc'd after a while, presumably after 65535 reductions of the
process that stored it (if that makes sense). My question is how can I
reduce the fullsweep_after or explicitly force a gc on the heap where it is
stored.

//Tom.



On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette <zeno490@REDACTED>wrote:

> A bit hard to understand what is happening without seeing f(A)'s code.
> AFAIK, even if you don't keep a reference to a binary, if you keep a
> reference to a binary created from it, it'll keep it in full in memory.
> ie:
> A = << some large binary>>.
> <<B:32/binary, _/binary>> = A.
> If you keep no references to A but you do to B, A will remain in memory
> because B will have been created as a pointer offset from A (as an
> optimization).
>
> Nicholas
>
>
> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly <ttom.kelly@REDACTED> wrote:
>
>> Hello List,
>>
>> I need some help on garbage collecting binaries.
>> I have an application that handles large binaries and under load it eats
>> up
>> all the available memory then falls over, even when I start all the
>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}".
>>
>> Reading point 5.15 on the page:
>> http://www.erlang.org/faq/how_do_i.html
>> leads me to think that calling the garbage collector on the process that
>> created a binary will clean it up but my shell experiment below shows me
>> I'm
>> wrong. It's now 10 minutes since I've done this experiment and the large
>> binary is still in memory.
>>
>> Maybe I have to call the garbage collector of the process whose heap is
>> storing the binary? If so, which process is it?
>>
>> Any pointers as to what I'm doing wrong or mis-understand will be greatly
>> appreciated.
>>
>> I'm on a legacy R12B5 system.
>>
>> //Tom.
>>
>>
>>
>> 18> process_info(self(),total_heap_size).
>> {total_heap_size,4181}
>>
>> 19> A = L(100000).
>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118,
>>  180,121,181,160,124,244,140,169,215,31,82,43|...]
>>
>> 20> process_info(self(),total_heap_size).
>> {total_heap_size,1149851}
>>
>> 21> f(A).
>> ok
>>
>> 22> erlang:garbage_collect().
>> true
>>
>> 23> process_info(self(),total_heap_size).
>> {total_heap_size,3194}
>>
>>
>> 24> memory(binary).
>> 6536
>>
>> 25> A = B(100000).
>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181,
>>  218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>>
>>
>> 26> memory(binary).
>> 106704
>>
>> 27> f(A).
>> ok
>>
>> 28> memory(binary).
>> 106704
>>
>> 29> erlang:garbage_collect().
>> true
>>
>> 30> memory(binary).
>> 106560
>>
>
>