[erlang-questions] Bounded Buffer Problems/Solutions
Thu Oct 11 01:35:28 CEST 2012
On 10/10/2012 12:37 PM, Eric Newhuis wrote:
> What are the generally accepted solutions for when producers need to be throttled? gen_server as consumer or otherwise?
> Are there any good solutions or philosophy for the bounded buffer problem in Erlang? ..flow control? Freaky out-of-band communication at a distance to limit messages sent? Use the idea of rope: Message in reverse to a producer shim that he is allowed to send N more messages before he needs to ask for the right to send more? Other?
> Are there common Erlang idioms for monitoring execution time at the gen_server level so that an API can respond upward with a policy of "the system is currently busier than a one legged man in a kicking contest?"
I haven't seen a generic chunk of Erlang code that would handle all throttling needs because I think it depends on the situation, so the source code is generally ad-hoc. However, I do think that in the situation that many Erlang process act as producers with 1 or more Erlang process acting as a consumer, you can use the process dictionary in a valid way (that most people wouldn't be too angry about). If you use a unique atom as the process dictionary key in each producing process that contains a timestamp/count tuple, you are able to do a simple rate limiter that is non-intrusive to the other source code. I have done this for async logging, to prevent flooding processes here:
Using the process dictionary helps to avoid checking with a message, in the consuming process. Also, checking in the producing process could be erroneous if all producing cases are not handled properly. Throttling generally wants temporary storage which makes it a problem better suited for the process dictionary than ets (and you can avoid global locking on ets tables).
More information about the erlang-questions