[erlang-questions] Twoorl: an open source Twitter clone

[Scott Lystig Fritchie]

Steve Davis <steven.charles.davis@REDACTED> wrote:

sd> All in all, the fundamental application scope of Twitter simply
sd> *screams* Message Queue at me. I'm not sure why the "experts" that
sd> Twitter have scavenged from IBM and Google haven't come to that
sd> conclusion also. Since Twitter have appeared to commit to a FS
sd> approach, I guess we'll have to see if future history proves me
sd> incorrect :)

In the serious, serious business of arm-chair architecting, I'd say that
a file system approach is almost certainly more scalable than their
current implemention.

As for "message queueing", there may be a misunderstanding over how MQ
systems typically work: they have producers *and* consumers, and (more
importantly) consumers actually "consume".  Consuming a queue item
usually means also deleting it from the queue.  A single Twitter user X
can have thousands of consumers all trying to consume the same messages,
but in a typical MQ system, all but the first consumer would find X's
queue empty.

For one example, see the RabbitMQ FAQ, "Q. How do I archive to RDBMS?".

-Scott