[erlang-questions] On selective receive (Re: eep: multiple patterns)

[Sean Hinde]

On 3 Jun 2008, at 12:30, Chandru wrote:

>
> We already have a mechanism to restart if a queue grows too large  
> (actually 2 - process_info monitoring, and out of memory !)
>
>
> I agree it is nearly impossible to predict this -  but what options  
> does a programmer have without this bounded queue facility.

Well, I guess, mostly you need to have a design that doesn't lead to  
massive queue build up under sustained overload :-). This might mean  
input load regulation, or tweaking the process structure (the logger  
process problem).

The system is unlikely to be performing to spec during this whole  
period of queue build up followed by cyclic restart - it doesn't  
really matter if the system restarts because it runs out of memory or  
cyclic restarts one process inside. It is still an outage for  
customers of the system.

All you need to know is that it has crashed and why, so you can fix  
the bug. The erl_crash dump will tell you about the huge message queue.

>   1. Introduce message queue monitoring for every process which is  
> potentially long lived, which imho is extra boiler plate code which  
> reduces readability of core functionality. Also there will be  
> different ways of doing it depending on how your process is  
> structured (gen_fsm, gen_server, gen_event, pure erlang...). If all  
> that one does upon detecting this condition is clear the message  
> queue by discarding messages, or terminate the process, wouldn't it  
> be good to have this built-in?

Another option - fix the system so that it doesn't get into that state.

>   2. have another process which monitors the entire system - which  
> is not very scalable when you have hundreds of thousands of processes.
>
>   3. Wait for the system to crash in live and then figure out what  
> happened.

Exactly. It is a bad bug that leads to such queue build up. Crashing  
is fine in this case, and probably preferable to lingering onwards  
silently failing to provide service.

Cheers,
Sean