Mnesia not suitable for time series storage?

[Ulf Wiger]

Den 2005-07-13 14:22:21 skrev Thomas Lindgren <thomasl_erlang@REDACTED>:

>
>
> --- Joel Reymont <joelr1@REDACTED> wrote:
>> My two requirements for tick storage are very
>> high-speed writes and
>> high-speed sequential retrieval.
>
> Assuming that you write as the ticks arrive, this
> sounds like a good fit for a plain old file per
> stream. So maybe mnesia is overkill?
>
> OK, this doesn't answer your question, I know :-)

I was also thinking along the lines of having a
load-balancing front-end that uses disk_log to
store ticks persistently. One could switch logs
in order to get a reasonable chunk in each.

The log could be stored on a RAID disk, providing
redundancy.

The load balancer could then forward ticks to a
diskless cluster, where ticks are possibly stored
in non-replicated, ram-only fragmented tables.

A problem would be to 'garbage-collect' the cluster,
removing ticks that are deemed as 'too old'.

Or perhaps this gets a lot easier if one uses ets
tables instead of mnesia. Keeping a list of ets
tables, and the age of objects in each, one can
simply drop tables that are too old. Problem here
is that we end up writing our own distributed database.

/Uffe