[erlang-questions] High volume CDR analysis

[Christian S]

On Jan 21, 2008 5:53 PM, Ukyo Virgden <listproc@REDACTED> wrote:
> Hi Christian,
>
> You're right. What I'm imagining is to collect call detail records
> from several telco equipment and periodically create reports. At this
> moment I'm not thinking about real-time (by realtime I mean as-it-
> happens) reports.
>
> So this basically means, collect input data in parallel, apply some
> transformation, store in mnesia and run a job to create reports.
>
> Therefore, the only data I need to store is for only one period,
> which is 100-300 million records of input.
>
> Any suggestions? I suppose there is a storage limit of 2gig for
> mnesia per node right?

There is a storage limit per disk table. I.e. you could create and
direct logging
to a new table before the current one has time to grow full.

I believe you would be better off recording these CDRs to flat files,
perhaps using
the disk_log library which can give you copies on multiple nodes.

The later would be a very primitive form of column-based table representation.