[erlang-bugs] Loss of sharing

[John Hughes]

Try running hog:main() in the program below. It creates a data structure in
one process, measuring heap size, and how it changes during the creation.
Then it sends it to another, measuring the memory allocated in the receiving
process. Here's the output:

 

Creating tree:: allocated 0 bytes, heap size 987 bytes

Receiving tree:: allocated 116769407 bytes, heap size 116769640 bytes

true

 

That is, the structure is less than 1K bytes in the sender, but over 100MB
in the receiver.

 

The reason is that it contains  a lot of sharing, and sharing is lost when
messages are sent to other processes. Sharing is ALSO lost when values are
copied to a new process being spawned. In fact, I've found no way to copy
structures between processes WITHOUT losing sharing. This means that some
data simply cannot practically be sent from one process to another.

 

I understand, of course, that if I send the same structure IN TWO SEPARATE
MESSAGES, that I am bound to get two copies in the receiver. But if I send a
structure in  ONE message, then there's in principle no reason why the copy
could not preserve sharing. That's true even when copying between nodes.

 

In my real application the data is funs, constructed in a complex manner so
that I cannot predict the sharing. When the sharing is lost, they get bigger
than the tree in my example. If I can't preserve sharing when I send them to
another process, then I can't structure my program using processes. That, in
turn, limits my ability to trap exits.

 

Any chance of introducing a sharing-preserving copy in message send and
spawn? It would be really useful!

 

John Hughes

 

 

-module(hog).

-compile(export_all).

 

% Builds a binary tree of depth N. Note the sharing!

tree(0) ->

    leaf;

tree(N) -> 

    T = tree(N-1),

    {T,T}.

 

measure(S,F) ->

    {heap_size,Before} = process_info(self(),heap_size),

    Result = F(),

    {heap_size,After} = process_info(self(),heap_size),

    io:format("~s: allocated ~w bytes, heap size ~w bytes~n",

                                   [S,After-Before,After]),

    Result.

 

main() ->

    T = measure("Creating tree:",fun() -> tree(25) end),

    Child = spawn(fun() -> child() end),

    erlang:yield(),

    Child ! T,

    erlang:yield().

 

child() ->

    measure("Receiving tree:",fun() -> receive Tree -> ok end end).

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20071106/c2eea99f/attachment.htm>