Mnesia data capabilities

[Chandrashekhar Mullaparthi]

Hi Daniel,

We store about 15 million records in mnesia. This is how it is.

The whole table is split across two nodes. The customer table is split into
128 fragments, and there are 64 fragments in each node. To minimise the
effect of partitioned networks, both nodes reside on the same physical
hardware. Mnesia on these two nodes is replicated to two nodes on another
machine.

Each node takes up about 1.5GB physical memory. It has been quite stable -
it has been up and running since February last year.

cheers
Chandru

On 30/12/03 1:12 am, "Inswitch Solutions - Erlang Evaluation"
<erlang@REDACTED> wrote:

> Hi,
> 
> I would appreciate if anybody can give us some advice about mnesia, as we have
> seen certain limitations, in terms of mnesia tables and data size, which we
> can not precisely know. So, we wonder if there is an optimum relationship
> among the number of tables and the size of each table for a certain amount of
> records, of a certain size to be saved.
> 
> Let me describe you our situation: we would like to migrate 10.000.000
> records, from a file server to a mnesia database:
> 
> The information on the file server, is in text format, (each line is a record
> of 100 bytes) as follows:
> 
> It contains a complex directory structure, with 100.000 files, and each of the
> files has 100 lines of 100 bytes each. As I mentioned, each line is one
> ³record² with phone user data. Initially, all we want to do is to remove the
> file server, so we are not creating a set of fields for each record. Instead,
> we are considering each record as a single piece of information.
> 
> So, my question is how to build/distribute a set of tables in a ³mnesia safe
> structure² that could hold this amount of records (10.000.000 records) in an
> optimum configuration
> 
> We have initially considered the following alternatives:
> 
> 1)      Creating 1000 tables, with 10.000 records each. Tables have been
> created, but it crashes with ³mnesia info², with the tables empty
> 2)      100 tables, with 100.000 records each
> 3)      One huge table (10.000.000 records) partitioned
> 
> 
> Best Regards,
> 
> daniel
> 
>