[erlang-questions] Priority Queue

[Eranga Udesh]

Thanks Jay, I got your point. 

Since most of the times the system is in controlled environment (eg.
AXD301), I think the messages originating are all (or most) pre-known. Its
unlikely spam is coming, but giving priority to control messages over data
messages, multi-level priority data messages, etc., is most concerns.

BRgds,
- Eranga


-----Original Message-----
From: Jay Nelson [mailto:jay@REDACTED] 
Sent: Sunday, July 08, 2007 12:27 AM
To: Eranga Udesh
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Priority Queue

> However using a router process, I still don't see the priority  
> queue problem
> resolved, other than using a selective receive.

The router _must_ unload all messages.  In your example, you have 2  
priorities so create two instances of queue, receive AnyMsg handle it  
as a worker Ack or put it in the correct queue.  The trick is to make  
the recursive loop for next message smart enough to pass on messages  
so that work is done as we are receiving more messages, rather than  
just unloading all first because in a continuous environment you may  
never hand anything to a worker and will run out of memory.

The key to using a router is to _not_ use selective receive at all.   
You are simulating a mailbox and distribution rules in erlang, so the  
entire message queue must be represented as erlang data structures in  
the router process.

>
> It's like "Importance" flag in Email. I

I thought the answering machine example was self-expalantory, but  
maybe it was a bad example.  In the case of email, every spammer  
sends me high priority messages.  If I look at them first, I have  
ceded control of my time to others who are trying to manipulate my  
attention span for their own gain.

This is the common approach to "performance tuning" as well.  Given a  
particular problem spot in a system, tweak the approach to favor the  
problem I am trying to solve right now.  Over time you end up with a  
bunch of localized hacks that collective make the system  
unpredictable and inconsistent.

It is fundamental, priority is in the eyes of the receiver only.  The  
sender has no sense, knowledge or standing as to whether a message is  
higher priority than any other.  A decision today on priority will  
necessarily change when a new process or new message types are added  
to the system.  You should not architect a system which depends on  
knowledge of that sort being distributed among many components -- you  
will encounter a policy change that is extremely difficult to make.   
The purpose of the router is not only to solve an erlang  
implementation challenge.  It also isolates all knowledge of  
prioritization (which may be dependent on adaptive operational  
considerations) so that there is a separation of concerns.

jay