[erlang-questions] On selective receive (Re: eep: multiple patterns)

[Chandru]

2008/6/3 Sean Hinde <sean.hinde@REDACTED>:

>
> On 2 Jun 2008, at 14:31, Chandru wrote:
>
>  2008/6/2 Mats Cronqvist <mats.cronqvist@REDACTED>:
>> Ulf Wiger wrote:
>> > I would really like to discourage people from avoiding
>> > selective receive because it's "expensive". It can be
>> > expensive on very large message queues, but this is
>> > a pretty rare error condition, and fairly easily observable.
>> >
>>
>>  i think the "issue" of how the emu deals with huge in-queues is pretty
>> uninteresting.
>>  in my personal experience, every single time this has come up the real
>> problem has turned out to be lack of proper flow control (typically
>> using {active,true} sockets). having 100k messages in an in-queue is not
>> a realistic use case.
>>  the fact that this is not, afaik, particularly well documented is of
>> course a problem.
>>
>> This is true - but if one has no prior experience of this situation, it is
>> hard to understand why a system is behaving sluggishly. What will be nice is
>> having an option, as Ulf suggested earlier, to have bounded message queues
>> (kill the process if the message queue length exceeds a certain value). That
>> way, flow control problems will be more readily visible to users. In real
>> life situations, when a process gets into this state, the only way to fix it
>> is to kill that process as it will probably never catch up. This has been
>> discussed before:
>> http://www.erlang.org/pipermail/erlang-questions/2006-January/018364.html
>>
>> It also fits in well with the "Let it crash" philosophy.
>>
>
> I respectfully disagree. It is nigh on impossible to predict where there
> might be some error that leads to a large queue, and this would lead to
> "defensive programming" where every process has a short max length. This
> would result in random crashes and loss of data for those uncommon
> situations in an generally well designed system where there might be a
> legitimate short term peak in queue lengths.
>
> We already have a mechanism to restart if a queue grows too large (actually
> 2 - process_info monitoring, and out of memory !)
>
>
I agree it is nearly impossible to predict this -  but what options does a
programmer have without this bounded queue facility.

  1. Introduce message queue monitoring for every process which is
potentially long lived, which imho is extra boiler plate code which reduces
readability of core functionality. Also there will be different ways of
doing it depending on how your process is structured (gen_fsm, gen_server,
gen_event, pure erlang...). If all that one does upon detecting this
condition is clear the message queue by discarding messages, or terminate
the process, wouldn't it be good to have this built-in?

  2. have another process which monitors the entire system - which is not
very scalable when you have hundreds of thousands of processes.

  3. Wait for the system to crash in live and then figure out what happened.

cheers
Chandru
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080603/e8aceffb/attachment.htm>