[erlang-questions] Market data, trading strategies and dropping the last element of a list

[Joel Reymont]

Folks,

I'm building me a trading platform. I would like to collect market  
data that flows in really fast. I'm envisioning trading strategies as  
processes that take a quote and emit a buy, sell, etc. message.

Each trading strategy would need to keep the last N quotes for  
several securities (MSFT, IBM, GOOG, etc.) and this N can be up to  
several hundred. Ideally, strategies would subscribe to quotes and  
specify how many quotes they want to keep in the buffer. I would then  
supply functions to retrieve a quote up to M quotes back. This way  
you could do close('MSFT', 100) to retrieve the closing price of MSFT  
100 quotes back.

The problem I have is what data structure to use for quote history  
accessible by the strategies. There are 3 basic operations on the  
history buffer:

1) Collect up to M quotes in the buffer
2) Push a new quote into the buffer and drop the last quote (push it  
out)
3) Access a quote up to X quotes back

I would like to be very efficient here as strategies would need to  
emit buy/sell orders within milliseconds. This would very much be a  
soft real-time system.

I will likely store market data in disk_logs. I suppose I could  
somehow slide through binaries that I'm storing in the disk_log but I  
would like to insulate trading strategies from having to deal with  
parsing of binaries.

Uffe wrote "sliding tuple window" code a while back but it requires  
modifying Erlang to add extra bifs. I suppose I could try using a  
regular list for this but I can't figure out how to drop the last  
element.

Any suggestions?

	Thanks, Joel

--
http://wagerlabs.com/