[erlang-questions] gen_server aggregating calls/casts

[Peer Stritzinger]

On 2013-07-09 17:23:34 +0000, Jonathan Leivent said:

> My gen_server can actually tolerate having incoming requests dropped 
> more than it can tolerate having aggregation use up lots of memory 
> (potentially dying if it runs out, or at least slowing things down). 
> Maybe that means I will need to use gen_UDP - I knew I would eventually 
> switch to UDP to bypass the TCP overhead, but now that looks like a 
> sooner-rather-than-later project.

But with gen_tcp you'd use {active, once} anyway (yes?) and this would 
give you TCP's flow control.

> For now, I cheated and used gen_server's internal message format - just 
> because doing that turns out to perturb the code I already have the 
> least.  All it takes is:
> 
> aggregate_requests(L) ->
>    receive
>      {'$gen_call', From, R={request, _}} ->
>        aggregate_requests([{From, R}|L])
>    after 0 -> L
>    end.
> 
> That's just too tidy and easy to pass up, even if it does break 
> modularity.  I know I'll have to replace this when I face the 
> flow-control issue.

I often just (mis)use the gen module, especially gen:call(), this gives 
you all the neat monitoring and everything from gen_server.  But then I 
use a plain erlang server (started with proc_lib:spawn_link or even use 
something like Ulf's plain_fsm) to be able to do selective receive.

Selective receive is a very powerful tool in Erlang that you can't use 
properly if using the gen_* servers.  Those are for sequential 
requests, and often you want this.  But if you need selective receive 
as in your case they just get into the way.

I think gen:call should be documented and made official so it is 
exposed better and can be used with a more quiet state of mind.  On the 
other hand I don't think its likely to change so I use it anyway.

Having said this, your possibly unbounded growing mailbox is a bit 
scary and will at some point probably crash your node.  You should 
combine your selective  receives with some way of limiting requests.

Jespers https://github.com/jlouis/safetyvalve comes into mind, I think 
it has pluggable queuing so you might be able to keep your mailbox 
selective recieve aggregation and just use Safetyvalves CoDel to 
control it as queue.

@Jesper would that be possible with Safetyvalve (to use it with a 
Erlang mailbox as Queue)?


> 
> -- Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130710/162c1fa0/attachment.htm>