[erlang-questions] understanding the scaleability limits of erlang and mnesia
Wed Jan 27 02:40:13 CET 2010
Disk and Network activity are good.
In fact, after I made Paul M's change to no_table_loaders (raise to 20), I
see the network interface pretty much peg out at 12 MB/s. Doing the math,
this hits 96 Mbs which is pretty good for a 100Mbs interface.
I guess at this point, I've got to go to a gigabit interface to improve
startup time / throughput ? As it stands, my app takes about 7 minutes to
startup. 5 of those minutes are lost in mnesia startup.
On Tue, Jan 26, 2010 at 2:33 PM, Paul Fisher <pfisher@REDACTED> wrote:
> You misunderstood me... uniqueness is good. Non-unique keys is bad.
> And for disk_copies tables it is effectively inserting into ets tables,
> yes. Extremely, non-unique keys really slows down loads of tables at
> startup. You should be able to test this by monitoring disk and network
> i/o while it is starting the tables. If both are low, but while it
> takes forever to start you table(s), then the keys are probably the
> On Tue, 2010-01-26 at 16:23 -0600, Brian Acton wrote:
> > I understand that inserts would take longer in the former example than
> > in the latter. However, I don't understand why this would so greatly
> > impact restart / recovery time ? Is this because mnesia is effectively
> > reinserting the records, rather than blatting the file from another
> > node over the wire ? Presumably, the latter would be much much
> > faster...
> > Any references (besides what is on erlang.org) that you use that
> > details this info ?
> > Also, rewriting my app to lessen the uniqueness of the indexes is
> > virtually impossible. I'm dealing with user ids which are inherently
> > unique.
> > --b
> > On Tue, Jan 26, 2010 at 12:50 PM, Paul Fisher <pfisher@REDACTED>
> > wrote:
> > On Tue, 2010-01-26 at 14:19 -0600, Brian Acton wrote:
> > > Can you explain more about how index uniqueness affects
> > recover /
> > > startup times ?
> > Fundamentally, this comes down to the way that ets tables
> > work... Try to
> > insert 100000 records with unique index values into a table,
> > then try to
> > do 100000 records with either the atoms 'true' or 'false'.
> > You will see
> > the problem in the runtime results.
> > --
> > paul
More information about the erlang-questions