[erlang-questions] Recursive Heap Allocation

[Richard Carlsson]

On 07/25/2012 09:36 AM, Joe Armstrong wrote:
> This program has similar behaviour
>
>    -module(bug1).
>    -compile(export_all).
>
>    test(N) ->
>            io:format("~p~n", [N]),
>            test(N+1).
>
> I  did this:
>
> 1 > spawn(fun() -> etop:start() end).
> 2> bug1:test(0).
>
> You can see that user_drv global process is accumulating messages
> faster than it can handle them.
> It's in usr_drv:io_request/3.
>
> As far as I'm aware io:format did have flow control so this shouldn't happen

The I/O protocol has no built-in flow control.

> Now I ran with "erl -smp disable"
>
> The counter stays at zero forever.

With only a single scheduler, it will be alternating between running the 
producer process and the consumer processes, but they won't be running 
at the same time, so the consumer has a better chance of keeping up. The 
number of reductions used per message is the deciding factor, and if I 
recall correctly, there's a built-in penalty in reductions when you pass 
a message to a process that already has a long queue.

With multiple schedulers, the producer and the consumer will run on 
separate schedulers, i.e., they run on separate OS threads and reduction 
count doesn't matter, only actual time per message. The only thing 
keeping them back a little is the synchronization needed for the 
mailbox. It's not so surprising that it's easier to overload the 
consumer in this case. (Also note that the I/O protocol causes the 
formatting to be done on the receiver side, so if you're printing 
complex terms, you're making the consumer do a lot more work per message 
than the producer.)

     /Richard