[erlang-questions] High volume CDR analysis

Ukyo Virgden listproc@REDACTED
Mon Jan 21 17:53:10 CET 2008

Hi Christian,

You're right. What I'm imagining is to collect call detail records  
from several telco equipment and periodically create reports. At this  
moment I'm not thinking about real-time (by realtime I mean as-it- 
happens) reports.

So this basically means, collect input data in parallel, apply some  
transformation, store in mnesia and run a job to create reports.

Therefore, the only data I need to store is for only one period,  
which is 100-300 million records of input.

Any suggestions? I suppose there is a storage limit of 2gig for  
mnesia per node right?

Thanks in advance,

On 15.Oca.2008, at 21:20, Christian S wrote:

> I spent some time googling these acronyms, and as far as I could
> figure out it is:
> CDR:  Caller data record
> ETL: Extract, transform, load
> My guess is that Ukyo gets vendor specific report formats from some
> telecom equipment
> and need to rectify all to one format that one can use to extract
> reports from, for billing
> and homeland insecurity.
> I'm not sure mnesia is a good fit. But then again I dont know what
> kind of system Ukyo
> imagined. I got the feeling he would want to store a hundred million
> CDRs in mnesia
> tables, that seems to require that you know how to rewrite the mnesia
> backend to add a
> sleepycat db table-type, then not release it because you think it
> wasnt done right.
> Having a high-availability cluster where each CDR is reported
> as-it-happens, updating
> an account's usage-data, seems quite doable. But having an actual log
> of transactions
> for each account is probably important.
> On Jan 15, 2008 7:42 PM, Ahmed Ali <ahmed.nawras@REDACTED> wrote:
>> Hi Ukyo,
>> Just wondering, what's ETL?
>> Best regards,
>> Ahmed
>> On Jan 15, 2008 3:52 PM, Ukyo Virgden <listproc@REDACTED> wrote:
>>> Hi,
>>> I'm working full-time on a system which involves a hundreds of
>>> millions of CDR records.
>>> Our current setup is a typical ETL infrastructure which NOBODY is
>>> happy about.
>>> After reading and experimenting with Erlang and Mnesia, wouldn't it
>>> be possible to do such ETL with Erlang and Mnesia?
>>> Any suggestions?
>>> Regards,
>>> Ukyo.
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions

More information about the erlang-questions mailing list