[erlang-questions] trading systems

[Andrew Thompson]

On Thu, Oct 01, 2009 at 05:04:12PM +0100, Joel Reymont wrote:
> Kenneth,
> 
> On Oct 1, 2009, at 4:50 PM, Kenneth Lundin wrote:
> 
> >It would be nice if you could explain what the system needs to do  
> >and after that
> >why you think it is a problem to store prices in an ETS-table.
> >[...]
> >Why are you talking about mmapped files? Please explain for a novice
> >in trading systems.
> 
> It's high-frequency trading systems I'm talking about. Where's Serge  
> Aleynikov [1] when you need him?
> 
> I'd like to process an incoming price quote, make a decision and  
> submit a trade order with minimal latency. I know that this is  
> normally done using C++ (or OCaml [1]) but I'm wondering how this can  
> be achieved using Erlang.
> 
> Minimal latency is usually achieved by hitting the CPU cache as  
> frequently as possible and minimizing memory copying overhead. ETS  
> copies memory and I suspect it's use of the cache is rather poor by  
> virtue of being a "hash table" and pointing all over the place in  
> memory.
> 
> A simple array of floats or doubles groups data together in memory and  
> is very cache friendly. I figure that an Erlang binary is the closest  
> I can get to an array in Erlang. mmap-ing a file of floats or doubles  
> and wrapping it in a binary should avoid unnecessary copying of data  
> from disk to memory.
> 
> I think operating on binaries representing mmap-ed files of price  
> quotes is as close as you can get to low latency in Erlang. I know  
> that premature optimization is the root of all evil but I'd rather  
> have my teeth pulled than optimize Erlang. This is speaking from  
> experience. I figure a much better way is to choose a concept that  
> guarantees fast code from the start.
> 

There's always the undocumented hipe bit and bytearrays (which are
mutable) if you're willing to sacrifice everything on the altar of
performance.

http://erlang.org/pipermail/erlang-questions/2009-April/043460.html

Andrew