[erlang-questions] High volume CDR analysis
Tue Jan 22 10:52:56 CET 2008
On 21/01/2008, Ukyo Virgden <listproc@REDACTED> wrote:
> Any suggestions? I suppose there is a storage limit of 2gig for
> mnesia per node right?
There is a limit of 2GB per dets file table. Mnesia's disc_copies
tables and disc_only_copies tables are stored on disk in dets file.
That doesn't stop you from having multiple tables each with about 2GB
If you use 32 bit erlang, you can address 4GB of memory.
If you use 64 bit erlang, you can address more memory than you will
need to but will hit other design limitations in mnesia. A mix of
fragmented mnesia tables and 64 bit erlang will go a long way.
For 300 million records, at 1KB a record, you will roughly need 300GB
of storage. Since a machine with that much RAM is out of the question
for most people/organisations, you will have to process them
incrementally. That is where Joe's mail comes in about partitioning
More information about the erlang-questions