[erlang-questions] Support for newcomers and the popularity of Erlang

[Scott Lystig Fritchie]

Wojciech, sorry about wading into a thread so long after it started.

Ulf Wiger <ulf@REDACTED> wrote:

uw> OTOH, if all you do is simple get/put, and you need high throughput
uw> and excellent fault tolerance, riak will let you scale to levels way
uw> beyond what any sane person would attempt with mnesia.

My story about alternative distributed databases also starts with
Mnesia.  Once upon a time, I was asked to prototype a spam cannon that
would be capable of sending MMS & SMS messages to every mobile
subscriber in a Large Asian Country ... messages of a {cough} public
safety nature and manage and store and *not* lose message delivery
confirmations, to assist {cough} law enforcement ... in less than one
hour.

I had a prototype with Mnesia that could handle the workload, but only
if everything was perfect.  Once Mnesia started writing checkpoints of
its ETS-based tables to disk, throughput fell apart and could not
recover.  If a node failed during the peak workload, the cluster could
not survive the additional workload created by rerestarting the failed
node.

My solution was to write a distributed database that eventually became
Hibari(*).  And I now work at Basho on Riak(**).  Mnesia does some
really cool things, but large clusters (where "large" is more than 6 or
8 machines) are very tricky to administer, both for sustaining large
volumes of transactions and dealing with network partitions.

-Scott

(*) See https://github.com/hibari/hibari and
https://github.com/hibari/hibari-doc/tree/master/src/hibari

Fortunately, we didn't win the contract to build the spam cannon.

(**) See https://github.com/basho/riak