New trading systems platform
Ulf Wiger (AL/EAB)
Fri Jul 8 10:25:44 CEST 2005
HP Wei wrote:
> I do not use Erlang often enough to make comments on
> data storage using Mnesia.
> I just want to point out that to store tick data
> of many years needs a LOT of disk space plus
> the capability of compression (when writing)
> (and decompression when reading) in your database design.
Some form of archiving mechanism seems to be needed.
Re. compression, there is of course a very convenient
form available to erlang programmers:
Bin = term_to_binary(Data, [compressed]).
Decompression is transparently done by binary_to_term(Bin).
This uses LZW compression on the binary. Perhaps this is
not good enough?
Perhaps one could come up with a caching scheme in Mnesia,
where table segments can be converted from ram_copies
to disc_only_copies after a certain time? One can use a
custom fragmentation scheme that divides objects into time
periods. How many fragments could one have in one table?
More information about the erlang-questions