[erlang-questions] moving process state - copy on write - heap memory increase

[Björn-Egil Dahlberg]

Hi,

Sharing within a compounded data term is not preserved on send nor in
term to binary.

One way to reinstate sharing is to walk through the compounded term
and look for sharing (Wind3d uses this trick).

Ex.
Ls = <list of items>,
{Sh,_} = lists:mapfold(fun(I, T) ->
    case gb_trees:lookup(I,T) of
        none -> {I, gb_trees:enter(I,I,T)};
        {value, V} -> {V, T}
    end
end, gb_trees:empty(), Ls),
Sh.

(disclaimer: I am writing code when sleepy and I always forget the
mapfold syntax. But you get the idea.)

This approach requires some knowledge about the structure of the data
to be reshared. It will also temporary increase the total memory while
performing the resharing.

I'm sure there are other, perhaps much better, approaches as well to
this problem.

There exists several proposals on preserved sharing on send, but none
are currently implemented in the vm.

// Björn-Egil

Skickat från min iPad

11 sep 2012 kl. 23:45 skrev Roland <roland.koelbel@REDACTED>:

> Hello!
>
> I want to move the state of my process to another process. Unfortunately this is not possible without a huge increase in memory consumption as by copying the state the "copy on write semantics" are lost.
>
> This behaviour is demonstrated in my attached simplified t_process module.
>
> -> {ok, Pid} = t_process:start_link().
> -> t_process:bloat(Pid). --> memory consumption of the process is at 6657256 after "bloat" is finished
> -> t_process:reload_bag(P1). --> memory consumption is at 26295176 (4 times as much)
>
> No data has not been modified in any way, the data stays the same.
>
> My question is: Despite what i want to do with this state transfer, is there any way to copy the state of a process without losing the "copy on write semantics", so that the target process has the same memory footprint as the original one?
>
> Can someone point me to any direction/documentation on how i can get arround this issue?
>
> Thank you!
>
> -module(t_process).
>
> -behaviour(gen_server).
>
> -export([start_link/0, init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3, bloat/1, reload_bag/1]).
>
> -record(process, {bag = []}).
>
> start_link() ->
>    gen_server:start_link(?MODULE, [], []).
>
> bloat(Pid) ->
>    gen_server:cast(Pid, bloat).
>
> reload_bag(Pid) ->
>    gen_server:call(Pid, reload_bag).
>
> init([]) ->
>    {ok, #process{}}.
>
> handle_call(reload_bag, _From, State) ->
>    BinaryBag = erlang:term_to_binary(State#process.bag), % simulate state transfer / loss of semantics
>    io:format("Bag size is ~w.", [erlang:size(BinaryBag)]),
>    {reply, ok, State#process{bag = erlang:binary_to_term(BinaryBag)}};
>
> handle_call(_Call, _From, State) ->
>    {reply, ok, State}.
>
> handle_cast(bloat, State) ->
>    NewBag = lists:concat([State#process.bag, lists:foldl(fun(_C, List) -> List ++ [{"ABCDEFGHIJKLMNOPQRSTUVWXYZ"}] end, [], lists:seq(1, 50000))]),
>    io:format("Bloating finished."),
>    {noreply, State#process{bag = NewBag}};
>
> handle_cast(_Cast, State) ->
>    {noreply, State}.
>
> handle_info(_Info, State) ->
>    {noreply, State}.
>
> terminate(_Reason, _State) ->
>    ok.
>
> code_change(_OldVsn, State, _Extra) ->
>    {ok, State}.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions