[erlang-questions] understanding the scaleability limits of erlang and mnesia

[Paul Mineiro]

Hey it's an email by me from back in the day ...

http://www.erlang.org/cgi-bin/ezmlm-cgi/4/36604

In particular, the no_table_loaders setting is very important.

Also, setting master nodes on a table can elide the sync ... of course
this only works for static data, or data whose replication is managed in
another fashion, but it was a trick I found useful.

Also, if data is emphemeral in nature, local_copies is also a useful
trick.

-- p

On Tue, 26 Jan 2010, Brian Acton wrote:

> Hi Guys,
>
> I'm fairly new to erlang and I'm trying to understand better how erlang and
> mnesia deal with large scale. I'm wondering if anyone could provide some
> examples where they have been using erlang in a very large configuration
> (i.e. more than 10 machines  / more than 100 machines). I specifically am
> interested where people are running in a clustered configuration with an
> mnesia backing store to their application.
>
> It's been my experience that as much as a technology claims to be scalable,
> operability issues usually surface that make it bad in practice to simply
> just add more machines to the cluster. As an example, in my current
> configuration, I am experiencing a 10 minute mnesia recovery / verification
> time during node startup. If I try to bring up two nodes at the same time, I
> see even longer times and sometimes even failure during bring up. And my
> cluster is only four nodes in size. Of course, when the system is at steady
> state (i.e. all nodes up and running), it's awesome. However, when I have to
> go through a crash / recovery cycle, I usually want to shoot myself....
>
> Anyone got any war stories to share? Any papers or presentations that I
> should look at?
>
> Thanks muchly,
>
> --b
>