[erlang-questions] understanding the scaleability limits of erlang and mnesia

[Brian Acton]

Thank you. That is quite helpful. Our no_table_loader value is set to the
default 2. We'll start testing and raising the value.

I don't think that I can use the master node trick as our data is fairly
dynamic (basically a function of user activity - 25% of user base active per
day).

Is there reference to "local_copies" somewhere on the net. A quick search
for 'mnesia local_copies' did not turn up very much. Maybe, I'm not
understanding the reference correctly?

It's also good to hear from someone with a cluster > 5 nodes. I was sure
that people were running larger clusters but had not seen much activity on
this list regarding the topic.

Thx!

--b

On Tue, Jan 26, 2010 at 1:37 PM, Paul Mineiro <paul-trapexit@REDACTED>wrote:

> Hey it's an email by me from back in the day ...
>
> http://www.erlang.org/cgi-bin/ezmlm-cgi/4/36604
>
> In particular, the no_table_loaders setting is very important.
>
> Also, setting master nodes on a table can elide the sync ... of course
> this only works for static data, or data whose replication is managed in
> another fashion, but it was a trick I found useful.
>
> Also, if data is emphemeral in nature, local_copies is also a useful
> trick.
>
> -- p
>
> On Tue, 26 Jan 2010, Brian Acton wrote:
>
> > Hi Guys,
> >
> > I'm fairly new to erlang and I'm trying to understand better how erlang
> and
> > mnesia deal with large scale. I'm wondering if anyone could provide some
> > examples where they have been using erlang in a very large configuration
> > (i.e. more than 10 machines  / more than 100 machines). I specifically am
> > interested where people are running in a clustered configuration with an
> > mnesia backing store to their application.
> >
> > It's been my experience that as much as a technology claims to be
> scalable,
> > operability issues usually surface that make it bad in practice to simply
> > just add more machines to the cluster. As an example, in my current
> > configuration, I am experiencing a 10 minute mnesia recovery /
> verification
> > time during node startup. If I try to bring up two nodes at the same
> time, I
> > see even longer times and sometimes even failure during bring up. And my
> > cluster is only four nodes in size. Of course, when the system is at
> steady
> > state (i.e. all nodes up and running), it's awesome. However, when I have
> to
> > go through a crash / recovery cycle, I usually want to shoot myself....
> >
> > Anyone got any war stories to share? Any papers or presentations that I
> > should look at?
> >
> > Thanks muchly,
> >
> > --b
> >
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>