[erlang-questions] understanding the scaleability limits of erlang and mnesia
Tue Jan 26 23:58:53 CET 2010
On Tue, January 26, 2010 4:31 pm, Brian Acton wrote:
> Thank you. That is quite helpful. Our no_table_loader value is set to the
> default 2. We'll start testing and raising the value.
> I don't think that I can use the master node trick as our data is fairly
> dynamic (basically a function of user activity - 25% of user base active
> Is there reference to "local_copies" somewhere on the net. A quick search
> for 'mnesia local_copies' did not turn up very much. Maybe, I'm not
> understanding the reference correctly?
I believe the poster was referring to "local_content" tables. If you do
"erl -man mnesia" and search for local_content you will find:
* local_content When an application requires tables whose
local to each node, local_content tables may be used. The
the table is known to all Mnesia nodes, but its contents is
on each node. This means that access to such a table must be
locally. Set the local_content field to true if you want to
the local_content behavior. The default is false.
In using local_content tables the contents are not replicated across all
nodes which have the table.
> It's also good to hear from someone with a cluster > 5 nodes. I was sure
> that people were running larger clusters but had not seen much activity on
> this list regarding the topic.
We have some clusters of 12+ nodes running in production with much success.
There have been a number of tricks used over the years to make this work
reliable in the face of various recovery scenarios--if I get time later to
dig up some info on the specifics I'll post it to the list.
> On Tue, Jan 26, 2010 at 1:37 PM, Paul Mineiro
>> Hey it's an email by me from back in the day ...
>> In particular, the no_table_loaders setting is very important.
>> Also, setting master nodes on a table can elide the sync ... of course
>> this only works for static data, or data whose replication is managed in
>> another fashion, but it was a trick I found useful.
>> Also, if data is emphemeral in nature, local_copies is also a useful
>> -- p
>> On Tue, 26 Jan 2010, Brian Acton wrote:
>> > Hi Guys,
>> > I'm fairly new to erlang and I'm trying to understand better how
>> > mnesia deal with large scale. I'm wondering if anyone could provide
>> > examples where they have been using erlang in a very large
>> > (i.e. more than 10 machines / more than 100 machines). I specifically
>> > interested where people are running in a clustered configuration with
>> > mnesia backing store to their application.
>> > It's been my experience that as much as a technology claims to be
>> > operability issues usually surface that make it bad in practice to
>> > just add more machines to the cluster. As an example, in my current
>> > configuration, I am experiencing a 10 minute mnesia recovery /
>> > time during node startup. If I try to bring up two nodes at the same
>> time, I
>> > see even longer times and sometimes even failure during bring up. And
>> > cluster is only four nodes in size. Of course, when the system is at
>> > state (i.e. all nodes up and running), it's awesome. However, when I
>> > go through a crash / recovery cycle, I usually want to shoot
>> > Anyone got any war stories to share? Any papers or presentations that
>> > should look at?
>> > Thanks muchly,
>> > --b
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
More information about the erlang-questions