[erlang-questions] Erlang and Memory management

[Wenqiang Song]

A related question, for code like following:
test(<<255, T/binary>>) ->
    ok;
test(<<_, T/binary>>) ->
    test(T).

Does Erlang copy the input binary for each recursion? If the input is huge,
it would be very low efficient. If so, what should we do?

Thanks
Andy

On Wed, Jun 1, 2011 at 3:28 AM, Paul Meier <pablo.a.meier@REDACTED> wrote:

> I'm in a similar situation.  I've got a few large tries, implemented
> with gb_trees.  I keep them in an orddict, which I'm using as the
> State variable of a gb_server.  For example:
>
> ------
> init() ->
>    %% State is an orddict of 3 giant Tries, pre-made and fetched from
> the filesystem.
>    {ok, State}.
>
> handle_call({keyed_operation_on_large_data, Key}, _From, State) ->
>    GiantTrie = orddict:find(Key, State),
>    .... computations using GiantTrie....
>    {reply, Result, State}.
>
> I'm finding that my process is crashing due to inability to mmap, and
> the erl_crash.dump file tells me that the gen_server's supervisor (or,
> on occasion, the error_logger) is the one with a prohibitively large
> Stack+Heap.  Also, I never modify the Tries or the orddict containing
> them in calls to the gb_server: it's all read-only.
>
> Is storing the value in an orddict, or using regular terms as opposed
> to binaries causing unnecessary copying?  While _I_ know the data is
> being used in a read-only fashion, is there a place where copying
> might be occurring because the VM doesn't know this?
>
> I was hoping to play with binaries as a next attempt at fixing this
> (the links in this thread suggest that large binaries are handled more
> conservatively), but this thread appeared fortuitously, and maybe
> someone can spot the very likely obvious error that I'm making.
>
> I'm grateful for your help, whether in this thread or the others I've
> enjoyed reading ^_^
>
> -Paul
>
>
> On Tue, May 31, 2011 at 11:19 AM, Kostis Sagonas <kostis@REDACTED>
> wrote:
> > Robert Virding wrote:
> >>
> >> I just want to point out that these rules are BEAM specific and not
> Erlang
> >> specific.
> >
> > Well, this is not correct: the BEAM nowhere specifies that its
> instruction
> > set has to be implemented using a private heap architecture.  In fact, we
> > successfully used the BEAM instruction set to implement both a shared
> heap
> > and a hybrid heap implementation.  See:
> >
> >  http://user.it.uu.se/~kostis/Papers/heap.pdf
> >
> >> So on an implementation with a single heap all data is shared. Now there
> >> aren't currently many single heap implementations, to be exact only one,
> >> erjang (Erlang on the JVM). :-)
> >
> > There have been more.  We had an OTP system with a shared heap and the
> > hybrid heap system was part of OTP for quite a while.  IMO, it's too bad
> > that it was not maintained when Erlang/OTP was extended to support SMP
> > architectures.
> >
> > Also, the ETOS (Erlang to Scheme) system was based on a shared heap
> > architecture.
> >
> > Kostis
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110602/beddefc4/attachment.htm>