[erlang-questions] Re: A discussion in application design and event propagation

[Fred Hebert]

On Mon, Apr 19, 2010 at 4:20 PM, Garrett Smith <g@REDACTED> wrote:

> On Mon, Apr 19, 2010 at 12:24 PM, Fred Hebert <mononcqc@REDACTED> wrote:
> >
> >
> > On Mon, Apr 19, 2010 at 12:14 PM, Garrett Smith <g@REDACTED> wrote:
> >>
> >>
> >> Thinking in architectural terms, this strikes me as a pub/sub problem
> >> that a message broker would handle well. RabbitMQ (AMQP), e.g., would
> >> let you publish events to an exchange, which can then be routed to
> >> various queues via bindings. RabbitMQ in particular is a nice option
> >> for Erlang developers.
> >>
> >> Garrett
> >>
> >> ________________________________________________________________
> >> erlang-questions (at) erlang.org mailing list.
> >> See http://www.erlang.org/faq.html
> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >>
> >
> > I'm not too familiar with RabbitMQ and queue usage past using them to
> send
> > the load from an application's frontend to a backend (and also surviving
> > huge peaks in users).
> >
> > How well does it lend itself to real-time applications? How would you
> make
> > it work with N tables and 4N users? From my little experience with
> queuing
> > systems like this, it sounds like the number of queues or topics required
> > would be huge, otherwise the amount of filtering even higher. What's your
> > take on that?
>
> The overhead of relaying messages is very low. In a distributed system
> though you'll have network latencies as well as costs in decoding and
> encoding messages. I'm pretty sure this doesn't qualify as "real
> time", but for your use case it's probably fine. If you have specific
> requirements, I'd post to the rabbit list.
>

No specific requirement. I only picked the card game as a conversation
point. I only thought that this kind of pattern is something I'd encounter a
lot and wanted to discuss it and bounce ideas.


>
> I haven't paid close attention to your card game design, but at first
> stab, I'd use what AMQP calls a "topic" exchange. This delivers
> messages to queues based on routing key patterns. Some of the possible
> message keys might be:
>
>  blackjack.table.setup
>  blackjack.table.deal
>  blackjack.table.teardown
>  blackjack.player.join
>  blackjack.player.leave
>
> Message producers, e.g. a "table" process, would send messages to the
> exchange with a particular routing key designating an event. So if you
> spawned a new table process, you might send a message keyed with
> "blackjack.table.setup". The message would contain information about
> the table (number, capacity, etc.)
>
> Message consumers would each create a queue for receiving messages and
> "bind" that queue using routing key patterns -- think of this as
> subscribing to a message stream based on their keys. E.g. a player
> process is interested when a table deal event occurs -- it would bind
> using "blackjack.table.deal". You could "sniff" the messages for those
> applying to your table(s) or push the routing keys further by
> including table IDs (in which case you'd receive table events only for
> the tables you bind to -- e.g. "blackjack.table.4.deal").
>

Ah, I see. Thanks for the explanations. I had only briefly went over Queue
systems before (only as much as I needed to use them) and never had to play
with too complex pub/sub setups. This makes things clearer.


>
> As for the proliferation of queues, you'd need at least one queue per
> participating process. I think it's fair to say that rabbitmq will
> handily support tens of thousands of queues on a single machine
> (throughput aside), though this is also something you'd want to pass
> by the rabbit list. As always there are lots of variable when scale is
> an issue.
>
> But in general, message brokers specialize in routing messages
> involving lots and lots of producers and consumers. RabbitMQ happens
> to be one excellent option (there are certainly others) and has the
> benefit in your case of being very Erlang friendly.
>
> Garrett
>

This makes sense. Most of my [limited] experience with Queue systems were
with ActiveMQ, where I guess the number of queue is more limited.

I think RabbitMQ is a very interesting piece of architecture to add to an
application, but I'm not too sure about the latency part. I guess this is a
case of benchmarks, because as you said, it can be fine depending on the use
case.

Thanks for the explanation.