[erlang-questions] Twoorl: an open source Twitter clone

[Torbjorn Tornkvist]

Per Melin wrote:
> 2008/5/29 Yariv Sadan <yarivsadan@REDACTED>:
>> Facebook runs on MySQL -- that's pretty scalable :)
> 
> By Twitter's own account the database has been their bottleneck all along.
> 
>> The main reason I didn't go with Mnesia for storing most data is the
>> dets issues discussed previously on this list (specifically, the long
>> repair times and the need for fragmentation over 4gb). I use Mnesia
>> for storing session data though.
> 
> I wouldn't go for Mnesia either. If you're building something that's
> should possibly scale to the level of Twitter, the 2 GB limit just
> doesn't cut it, with or without fragmentation. And in the tests I've
> done DETS got very slow long before it hit the size limit. If you use
> disc_copies (which doesn't use DETS) instead of disc_only_copies
> you're of course instead limited by RAM.
> 
> I (like everyone else) have built a (web) chat service that I'll
> probably (alpha) release this weekend. The single biggest problem I've
> been wrestling with is how to persist the messages in an efficient
> manner. Right now I'm using term_to_binary and dumping that in a flat
> file (much like disk_log), which works for all my needs, except for
> full text search. I'd prefer not to build and store my own reverse
> index.

One approach could perhaps be to break out the DB-part of CouchDB and
make it available as a general purpose Erlang library ?

--Tobbe