[erlang-questions] avoiding overloading mnesia

[Paul Mineiro]

On Wed, 12 Aug 2009, Ulf Wiger wrote:
> Regarding [2], there are examples of Mnesia databases that
> have achieved very good scalability. It is not the best
> regarding writes/second to persistent storage, but as with
> [1], think about what your requirements are. Tcerl, just
> to name an example, gives much better write throughput, but
> requires you to explicitly flush to disk. Chances are that
> your data loss will be much greater if you suffer e.g.
> a power failure. Don't take this as criticism of tcerl, but
> think about what your recovery requirements are.

Totally accurate.  In particular we developed tcerl for a system that runs
distributed Mnesia on EC2.  When you are using Mnesia in distributed mode
you get a fresh copy of every table on startup by default, so trading off
local consistency across erlang VM crashes for speed was a good choice.
(In addition, on EC2, if you lose an entire instance, the local drive
state is not recoverable anyway).

So that's an example of what Ulf is talking about, thinking about your
recovery requirements.

-- p

p.z. Tokyocabinet has transaction support but I haven't done the work to
integrate it into tcerl.  I'm such an EC2 fan, it just hasn't come up.