hard limits?

[Ulf Wiger]

On Tue, 26 Oct 1999, Bob wrote:

bob>I'm mostly interested in the upper practical limits on number of
bob>nodes, and on individual Mnesia tables. Would it be hundreds of
bob>nodes, thousands, tens of thousands, or hundreds of thousands.

I am personally not aware of anyone running more than 70-80 nodes in one
erlang system. We are aiming for up to 32 in an *extremely* busy network.

It's difficult to reason about a practical upper limit, because it
obviously depends on what the applications are doing. One problem
is that erlang uses a fully meshed network. For a really big network, one
might want to have a hierarchical structure, and some kind of broadcast
mechanism, perhaps.

The global name server has been designed to handle larger networks, by
allowing multiple locker domains. This solves (?) one issue that we've had
before.

bob>Would Mnesia scale to tens of gigabyte (disk-based) tables, with
bob>millions of records?  Or, should Mnesia be limited to use as a near
bob>real time db, with some other backend for the big stuff?

I don't think mnesia scales that well in practise. A few problems:

- the table sync algorithm is a bit too naiive (it always copies the 
  whole table, instead of just the updates)
- the disk storage mechanism (dets) doesn't scale that well.

I don't see many hard limits that are so entrenched that they cannot be
lifted. Mnesia will surely evolve to handle larger data sets as users begin
to voice a requirement for it. Any hardcode database experts out there
who'd like to help? (:

/Uffe

Ulf Wiger, Chief Designer AXD 301
Ericsson Telecom AB                          tfn: +46  8 719 81 95
Varuvägen 9, Älvsjö                          mob: +46 70 519 81 95
S-126 25 Stockholm, Sweden                   fax: +46  8 719 43 44