[erlang-questions] Garbage collecting binaries

tom kelly ttom.kelly@REDACTED
Thu Oct 7 18:32:04 CEST 2010


Hi all,

Thanks for all your quick replies! Looks like my demonstration wasn't as
pure as I thought ;-)

I still have my original problem of ~75% of my VMs memory being taken up by
binaries, looks like I have a serious code-review ahead to see why this
isn't being freed up.

//Tom


On Thu, Oct 7, 2010 at 4:34 PM, Ulf Wiger <ulf.wiger@REDACTED>wrote:

>
> f(A) only unbinds the variable in the shell.
>
> The shell still has a history of commands and their results, so you
> would need to ensure that this is not taking up space as well.
>
> BR,
> Ulf W
>
> On 7 Oct 2010, at 17:07, tom kelly wrote:
>
> > Hi Nicholas,
> >
> > This was just run in the erlang shell, A was a 100000 byte binary,
> "f(A)."
> > told the shell to forget it, ie. set the reference count to zero. There
> were
> > no other references to A in this case nor were other binaries created
> from
> > it. It was gc'd after a while, presumably after 65535 reductions of the
> > process that stored it (if that makes sense). My question is how can I
> > reduce the fullsweep_after or explicitly force a gc on the heap where it
> is
> > stored.
> >
> > //Tom.
> >
> >
> >
> > On Thu, Oct 7, 2010 at 3:57 PM, Nicholas Frechette <zeno490@REDACTED
> >wrote:
> >
> >> A bit hard to understand what is happening without seeing f(A)'s code.
> >> AFAIK, even if you don't keep a reference to a binary, if you keep a
> >> reference to a binary created from it, it'll keep it in full in memory.
> >> ie:
> >> A = << some large binary>>.
> >> <<B:32/binary, _/binary>> = A.
> >> If you keep no references to A but you do to B, A will remain in memory
> >> because B will have been created as a pointer offset from A (as an
> >> optimization).
> >>
> >> Nicholas
> >>
> >>
> >> On Thu, Oct 7, 2010 at 10:07 AM, tom kelly <ttom.kelly@REDACTED>
> wrote:
> >>
> >>> Hello List,
> >>>
> >>> I need some help on garbage collecting binaries.
> >>> I have an application that handles large binaries and under load it
> eats
> >>> up
> >>> all the available memory then falls over, even when I start all the
> >>> data-handling processes with "{spawn_opt,[{fullsweep_after, 20}]}".
> >>>
> >>> Reading point 5.15 on the page:
> >>> http://www.erlang.org/faq/how_do_i.html
> >>> leads me to think that calling the garbage collector on the process
> that
> >>> created a binary will clean it up but my shell experiment below shows
> me
> >>> I'm
> >>> wrong. It's now 10 minutes since I've done this experiment and the
> large
> >>> binary is still in memory.
> >>>
> >>> Maybe I have to call the garbage collector of the process whose heap is
> >>> storing the binary? If so, which process is it?
> >>>
> >>> Any pointers as to what I'm doing wrong or mis-understand will be
> greatly
> >>> appreciated.
> >>>
> >>> I'm on a legacy R12B5 system.
> >>>
> >>> //Tom.
> >>>
> >>>
> >>>
> >>> 18> process_info(self(),total_heap_size).
> >>> {total_heap_size,4181}
> >>>
> >>> 19> A = L(100000).
> >>> [140,161,41,128,44,96,215,43,15,164,88,107,1,167,4,125,118,
> >>> 180,121,181,160,124,244,140,169,215,31,82,43|...]
> >>>
> >>> 20> process_info(self(),total_heap_size).
> >>> {total_heap_size,1149851}
> >>>
> >>> 21> f(A).
> >>> ok
> >>>
> >>> 22> erlang:garbage_collect().
> >>> true
> >>>
> >>> 23> process_info(self(),total_heap_size).
> >>> {total_heap_size,3194}
> >>>
> >>>
> >>> 24> memory(binary).
> >>> 6536
> >>>
> >>> 25> A = B(100000).
> >>> <<232,241,55,171,218,35,86,122,211,185,232,1,203,249,181,
> >>> 218,176,33,88,131,102,56,102,82,158,114,200,174,253,...>>
> >>>
> >>> 26> memory(binary).
> >>> 106704
> >>>
> >>> 27> f(A).
> >>> ok
> >>>
> >>> 28> memory(binary).
> >>> 106704
> >>>
> >>> 29> erlang:garbage_collect().
> >>> true
> >>>
> >>> 30> memory(binary).
> >>> 106560
> >>>
> >>
> >>
>
> Ulf Wiger, CTO, Erlang Solutions, Ltd.
> http://erlang-solutions.com
>
>
>
>


More information about the erlang-questions mailing list