[erlang-questions] High volume CDR analysis
Mon Jan 21 17:53:10 CET 2008
You're right. What I'm imagining is to collect call detail records
from several telco equipment and periodically create reports. At this
moment I'm not thinking about real-time (by realtime I mean as-it-
So this basically means, collect input data in parallel, apply some
transformation, store in mnesia and run a job to create reports.
Therefore, the only data I need to store is for only one period,
which is 100-300 million records of input.
Any suggestions? I suppose there is a storage limit of 2gig for
mnesia per node right?
Thanks in advance,
On 15.Oca.2008, at 21:20, Christian S wrote:
> I spent some time googling these acronyms, and as far as I could
> figure out it is:
> CDR: Caller data record
> ETL: Extract, transform, load
> My guess is that Ukyo gets vendor specific report formats from some
> telecom equipment
> and need to rectify all to one format that one can use to extract
> reports from, for billing
> and homeland insecurity.
> I'm not sure mnesia is a good fit. But then again I dont know what
> kind of system Ukyo
> imagined. I got the feeling he would want to store a hundred million
> CDRs in mnesia
> tables, that seems to require that you know how to rewrite the mnesia
> backend to add a
> sleepycat db table-type, then not release it because you think it
> wasnt done right.
> Having a high-availability cluster where each CDR is reported
> as-it-happens, updating
> an account's usage-data, seems quite doable. But having an actual log
> of transactions
> for each account is probably important.
> On Jan 15, 2008 7:42 PM, Ahmed Ali <ahmed.nawras@REDACTED> wrote:
>> Hi Ukyo,
>> Just wondering, what's ETL?
>> Best regards,
>> On Jan 15, 2008 3:52 PM, Ukyo Virgden <listproc@REDACTED> wrote:
>>> I'm working full-time on a system which involves a hundreds of
>>> millions of CDR records.
>>> Our current setup is a typical ETL infrastructure which NOBODY is
>>> happy about.
>>> After reading and experimenting with Erlang and Mnesia, wouldn't it
>>> be possible to do such ETL with Erlang and Mnesia?
>>> Any suggestions?
>>> erlang-questions mailing list
>> erlang-questions mailing list
More information about the erlang-questions