geometric memory growth

[Ulf Wiger (AL/EAB)]

Here's a subtle bug that caused me about one day's worth
of headache:

I have process that holds a gen_tcp session, and
multiplexes "virtual sessions" on top of it.

When running a benchmark, I noticed exponential
memory growth, and it took me a while to notice 
that the growth was actually taking place in the 
innocent benchmark loop (about 7 lines of code,
basically this:

vcc_rpc2(Channel) ->
    vccTransport:msg(Channel, 0, vcc_rex, {self(), ping},
		     [{temp_channel,true}]),
    receive
	{_, _, pong} ->
	    ok;
    end.

which amounts to an ets:lookup, a gen_server:call()
and waiting for a message where the third element is
'pong')

17 iterations made the VM allocate 1 GB of RAM.

The problem? (finally diagnosed once Björn G tipped
me off about erts_debug:size(Term) - thanks Björn!)

The second element of the message was a fun:

    SendF = fun(To, Msg) ->
	          msg(State#state.channel,
	          ChNo, To, Msg)
	      end,

Horrid, isn't it?

Writing it this way, causes the _whole_ State
to be bound inside the fun. And State of course
contained, among other things, the metadata for
the virtual sessions (e.g. other SendF instances.)

This wasn't much of a problem as long as everything
was local references on the same heap (it probably
would have shown after a while, though), but when 
the fun was passed in a message to another process,
all relative references were flattened out, and 
the size of the fun doubled in each iteration.


The obvious fix:

    Channel = State#state.channel,
    SendF = fun(To, Msg) ->
	          msg(Channel,
	          ChNo, To, Msg)
	      end,

Now, wouldn't it be great if the compiler
could figure out how to do this at compile-time?

/Uffe