Message Routing Paradigms

[Luke Gorrie]

(Apologies if this is a repost, I had a crash while sending..)

G'day Eric,

> I am aware of Erlang's messaging capabilities and yet there is still a
> general need for higher level protocols like the one's I mentioned.
> 
> Even the CosEvent module demonstrates acceptance of the decoupled
> publish/subscribe paradigm.
> 
> Do you or anyone else have any idea of messaging "middleware" developed
> with Erlang for Erlang that addresses message routing like the CosEvent
> module, but without the overhead of CORBA?

For basic decoupled publish/subscribe in straight erlang (plus some
nice features, like synchronous/blocking notification), there is the
gen_event behaviour. You can find details under the 'stdlib' part of
the manual:
http://www.erlang.org/doc/r9b/lib/stdlib-1.11.0/doc/html/index.html

In general though I think that Erlang leans on its lovely native
support for concurrent/distributed programming to let you roll your
own solution to the problem at hand, as oppossed to CORBA's focus on
generalized frameworks. But, Erlang does have generalized frameworks
for some things in the form of behaviours too (am I trying to have
have my cake and eat it too? :-))

Could you post some more details about your problem? You will probably
get some nice code in reply - it is rather fun to write this sort of
thing in Erlang, and usually there's a very nice and simple way.

As to 10,000 messages per second on 1ghz, that'd be 100 microseconds
per message. Some message passing benchmarks were recently posted here
under the "Design phase in Erlang" thread. I just ran Joe's benchmark
(http://www.sics.se/~joe/ericsson/du98024.html) on my 800mhz Athlon
and it says that sending small messages (an integer in this case)
takes about 2 microseconds and constant time (doesn't depend on number
of processes). That also includes the time to context-switch between
processes for each message, and there is a lot of that since the
processes (10,000 of them) were arranged in a chain and running in
lock-step.

In that case communication and synchronization should require about 2%
of your time and leave you 98us/message to do what you gotta do. Not
that this is conclusive or anything - you may want to send several
messages per event (e.g. for flow control), if your messages are large
you may want to switch to unified heap, etc etc. But it seems like a
promising first approximation (unless I've missed something
embarrassing :-))

Cheers,
Luke