Time series db: please help me optimize it

[Joel Reymont]

Folks,

This is my attempt at creating a time-series database in Erlang. I  
specifically want to store a day's worth of data in a separate disk  
log and keep the columns separate. You can read more on the whys  
here: http://groups-beta.google.com/group/uptick/

You can get the data from http://wagerlabs.com/ES4U.csv.gz. Run it  
like this:

tickdb:load("ES4U", "ES4U.csv").

I had a version that did not cache open disk logs but used tickdb:add  
directly, without going through a gen_server. It ran almost 3x faster!

Any suggestions on how to optimize this are appreciated. There are  
119,275 trade record in the CSV file. I divide the total run time by  
this number to get the average per insert. I would like to do at  
least 100 inserts per second.

The code is part of the open source trading platform that I'm creating.

     Thanks, Joel

--
http://wagerlabs.com/uptick


-------------- next part --------------
A non-text attachment was scrubbed...
Name: text.erl
Type: application/octet-stream
Size: 1624 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20050717/911496b3/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tickdb.erl
Type: application/octet-stream
Size: 4736 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20050717/911496b3/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tickplus.erl
Type: application/octet-stream
Size: 1168 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20050717/911496b3/attachment-0002.obj>
-------------- next part --------------