[erlang-questions] Twoorl: an open source Twitter clone

Steve Davis steven.charles.davis@REDACTED
Wed Jun 4 14:57:25 CEST 2008


Hi all,

This is indeed an interesting discussion, albeit totally orthogonal to
what I'm *supposed* to be focused on... :)

WRT to the use of MQ --- David Mitchell seems to have covered the
issue of persistent queues and thus points out - correctly to my mind
- that there's no need for RDBMS storage of tweet content, but then...

On Jun 3, 11:45 pm, "Yariv Sadan" <yarivsa...@REDACTED> wrote:
> I considered using a reliable queuing mechanism such as RabbitMQ or
> Amazon SQS but I don't think it would make the architecture inherently
> more scalable (more reliable maybe, but not more scalable).

Yariv, did you take a look at RabbitMQ clustering? It's claimed to be
near-linear (though I don't have practical experience of this, it
feels like it should be a valid claim)... http://www.rabbitmq.com/clustering.html
BTW, in case anyone is wondering -- i have no commercial interest or
connection with either lshift or cohesive.

Unlike Joe, I'm not concerned by the Web server (Yaws) scaling -- if
you were to get serious you'd probably shove a hardware load balancer
setup in front of your web servers and that would allow you to scale
httpd instances horizontally pretty much without limit (an example for
those interested in the HWLB would be Cisco's Content Services Switch
- http://www.cisco.com/en/US/products/hw/contnetw/ps792/ ) -- I'm
assuming here that the web server user session becomes decoupled from
the persistent tweet data by moving the latter out into the
(persistent queue) MQ cluster...

Also - I did do a bit of reading around after seeing Patrick Logan's
suggestion for using XMPP/Ejabberd as an alternative to MQ. It does
appear that it also could be a back-end candidate - and indeed Mickael
Remond has written on that issue directly
http://www.process-one.net/en/blogs/article/introducing_the_xmpp_application_server/
--- using XMPP is a less "traditional" approach than using an MQ, but
Mickael himself does present evidence that the idea has legs. So yes,
ejabberd does look like an alternative candidate to RabbitMQ that's
worth examining closely for anyone actually building a "Twitter done
right".

regs
/s



More information about the erlang-questions mailing list