Telecom & Mnesia - Call Detail Record Collection

[Chandrashekhar Mullaparthi]

We store information about our entire customer base (10+ million) in an
mnesia database. Right now, this is split into fragments across 8 machines
with replicas on 8 other machines. We are now planning to put the entire
database on one machine with the database split into fragments across two
erlang nodes. And our previous experience shows that we can support upto a
1000 transactions/sec.

But this database takes relatively few writes compared to the amount of
reads.

If you want to store a lot of data you should fragment your mnesia tables.
We haven't really thought of using mnesia for storing CDRs - my initial
thoughts would be that mnesia wouldn't be very suitable for this.

let us know how you get on.

cheers,
Chandru

-----Original Message-----
From: Martin J. Logan [mailto:martin@REDACTED]
Sent: 10 April 2002 21:57
Cc: 'Erlang Questions'
Subject: Telecom & Mnesia - Call Detail Record Collection


Hello All,
    I am working for a small telephony company here in the States. The only
method that we have for billing our customers is via call detail records.
The
system that we are using now is a rather poor one. My boss and I propose
that
we can revolutionize the way that we do this by employing erlang and mnesia.
All of my past projects with it have been extremly succesfull. I have
started
running some preliminary stress/load tests with mnesia and I am encountering
a
few problems. Mnesia really seems to slow down at 120,000 small records
stored
as disk_only_copies and at 300,000 small records stored as ram_copies.  The
requirements for this project could be in excess of 8 gig of data storage
with
500,000 records comming in on a daily basis. I have always thought of mnesia
as a real time db for storing smaller amounts of data. I would like to build
this cdr collection and reporting system entirly in erlang though and remove
the need for the sql db if possible. The more erlang the better off we all
are.
The quiestions I have are:

1. Has anyone stored such a vast amount of data in mnesia? I could not find
much on the list.
2. How does mnesia preform under large loads?  I need to do reporting on the
data.
3. What are the most effective mnesia configurations for maxing out its data
storage potential? What is its potential?
3. Has anyone out there used mnesia/erts/otp for call detail record
collection? If so can you give me any recomendations
    for my project?

                    Thank you for any help in this matter,
                    Martin.



NOTICE AND DISCLAIMER:
This email (including attachments) is confidential.  If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents.  We cannot accept liability for any breaches of
confidence arising through use of email.  Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions.  We will not accept responsibility for any commitments
made by our employees outside the scope of our business.  We do not warrant
the accuracy or completeness of such information.