[erlang-questions] gen_server aggregating calls/casts

[Jonathan Leivent]

On 07/08/2013 07:27 PM, Robert Raschke wrote:
> Have you considered using a queuing framework, like RabbitMQ?

What I'm trying to implement is too low level to make use of RabbitMQ. 
Ideally, if this works, it would be used within such frameworks.

> ...
>
> Alternatively, you'll probably want to double layer your server, as was
> previously suggested.

I think that will eventually be my best option - especially as I just 
learned that Erlang does no inter-node flow control on its own.  So, the 
outer layer will have to both aggregate requests AND flow control them 
as well, as there may be a very large number of requests coming from a 
very large number of clients.

It's surprising to me that Erlang provides no built-in flow control, 
since it is using TCP.  A gen_server that just receives and aggregates 
messages on a list could end up using a lot of memory with no way to 
push back.

My gen_server can actually tolerate having incoming requests dropped 
more than it can tolerate having aggregation use up lots of memory 
(potentially dying if it runs out, or at least slowing things down). 
Maybe that means I will need to use gen_UDP - I knew I would eventually 
switch to UDP to bypass the TCP overhead, but now that looks like a 
sooner-rather-than-later project.

For now, I cheated and used gen_server's internal message format - just 
because doing that turns out to perturb the code I already have the 
least.  All it takes is:

aggregate_requests(L) ->
   receive
     {'$gen_call', From, R={request, _}} ->
       aggregate_requests([{From, R}|L])
   after 0 -> L
   end.

That's just too tidy and easy to pass up, even if it does break 
modularity.  I know I'll have to replace this when I face the 
flow-control issue.

-- Jonathan