[erlang-questions] trading systems

[Robert Raschke]

On Thu, Oct 1, 2009 at 12:01 PM, Joel Reymont <joelr1@REDACTED> wrote:

>
> On Oct 1, 2009, at 11:58 AM, Robert Raschke wrote:
>
>  You'll want a column based data store, K comes to mind (
>> http://www.kx.com/).
>>
>
> I have this one and actually know how to use it.
>
>  If you want something a little bit more open, look at J (
>> http://www.jsoftware.com/) and it's JDB effort.
>>
>
> A nice free replacement.
>
>  Staying with the one letter theme (must be an in joke of stats heads,
>> methinks), there's also S and R. Good look searching for those ;-)
>>
>
> Searching for QuantMod is easier. That's a module that provides trading
> tools for R, including charting.
>
> Still, I'd like to know how this can be done with Erlang, as optimally as
> possible. And no, I don't think I want to use byte arrays, gb_trees or the
> array module ;).
>
>
>
Shooting from the hip, how about files containing Erlang binaries for fixed
index ranges. So, per column have a series of files col0, col1000000,
col2000000, ... If you're on 64bit, then mmapping whole columns without
segmentation might be an option (that's how J does it). Each column contains
fixed size data values. And then have an access layer with potential
caching. Speedy things needed would be the loading of a file to memory and
sequential access through the data. If live update is important, you'll want
zero cost append.

And over the Erlang binaries containing your fixed sized data you can
implement map and fold operations. Easy. <tongue firmly in cheek>

Robby