New trading systems platform

[Vlad Balin]

> > In case of ATS we need disk tick storage for the backtesting
> > purpose only. It implies very simple access scenario (rare
> > look-ups, sequential [forward in time] read of large data
> > volumes, "joins" of several tables by time field). It make
> > sense to develop file-based storage for tick storage (one
> > file per instrument, data stored sequentially by chunks of
> > compressed ticks), and avoid mnesia.
>
> Perhaps mnesia should be avoided... I'd still like to
> explore ways to keep it in the game though.  (:
Well, i would say mnesia should have BTree-based storage option then :),
and it should support clustered index. It should be able to work with
huge databases without tricks like partitioning single table into several
pieces.

Easiest way to achieve that would be to use Berkley DB
http://www.sleepycat.com/ as a storage ;)

> A final thought. Performance in disk-based accesses can
> be improved significantly if one can nail down the
> data format. This is something I think should be an option
> in mnesia.
Here's a bit different option. User might want to store data by chunks
in custom [compressed] format. Each chunk is being compressed separately,
and contains range of values. It would be helpful to do something to allow
this stuff
with chunks to be customized and accessed transparently, as if it would be
plain table with no chunks.