[erlang-questions] understanding the scaleability limits of erlang and mnesia
Igor Ribeiro Sucupira
Wed Jan 27 03:23:04 CET 2010
You might want to try this patch, which is in the branch of proposed updates:
If network bandwidth is your botteneck (and not disk I/O nor CPU), it
might help. It would also be a good test for the patch.
On Tue, Jan 26, 2010 at 11:40 PM, Brian Acton <> wrote:
> Disk and Network activity are good.
> In fact, after I made Paul M's change to no_table_loaders (raise to 20), I
> see the network interface pretty much peg out at 12 MB/s. Doing the math,
> this hits 96 Mbs which is pretty good for a 100Mbs interface.
> I guess at this point, I've got to go to a gigabit interface to improve
> startup time / throughput ? As it stands, my app takes about 7 minutes to
> startup. 5 of those minutes are lost in mnesia startup.
> On Tue, Jan 26, 2010 at 2:33 PM, Paul Fisher <> wrote:
>> You misunderstood me... uniqueness is good. Non-unique keys is bad.
>> And for disk_copies tables it is effectively inserting into ets tables,
>> yes. Extremely, non-unique keys really slows down loads of tables at
>> startup. You should be able to test this by monitoring disk and network
>> i/o while it is starting the tables. If both are low, but while it
>> takes forever to start you table(s), then the keys are probably the
>> On Tue, 2010-01-26 at 16:23 -0600, Brian Acton wrote:
>> > I understand that inserts would take longer in the former example than
>> > in the latter. However, I don't understand why this would so greatly
>> > impact restart / recovery time ? Is this because mnesia is effectively
>> > reinserting the records, rather than blatting the file from another
>> > node over the wire ? Presumably, the latter would be much much
>> > faster...
>> > Any references (besides what is on erlang.org) that you use that
>> > details this info ?
>> > Also, rewriting my app to lessen the uniqueness of the indexes is
>> > virtually impossible. I'm dealing with user ids which are inherently
>> > unique.
>> > --b
>> > On Tue, Jan 26, 2010 at 12:50 PM, Paul Fisher <>
>> > wrote:
>> > On Tue, 2010-01-26 at 14:19 -0600, Brian Acton wrote:
>> > > Can you explain more about how index uniqueness affects
>> > recover /
>> > > startup times ?
>> > Fundamentally, this comes down to the way that ets tables
>> > work... Try to
>> > insert 100000 records with unique index values into a table,
>> > then try to
>> > do 100000 records with either the atoms 'true' or 'false'.
>> > You will see
>> > the problem in the runtime results.
>> > --
>> > paul
"The secret of joy in work is contained in one word - excellence. To
know how to do something well is to enjoy it." - Pearl S. Buck.
More information about the erlang-questions