[erlang-questions] trading systems

[Jayson Vantuyl]

> True. I think I'll still start by experimenting with mmap-ed  
> binaries, or at least a binaries-based approach. These can always be  
> mmap-ed later.
Erlang assumes that binaries don't change.  Note that it never  
modifies them, only makes new ones.  Given that the compiler optimizes  
for this (or should), I would be careful about modifying binaries  
after they make it to Erlang.

> Yes, I may try porting HiPE to x86-64 when I have time to spare.
If you're very speed conscious, I would recommend running in 32-bit  
mode.  With everything I've done, it seems faster to page a GB of data  
in/out of memory than it does to deal with the larger pointers-- 
especially in something pointer-heavy (like a dynamic language).

> Let's consider a few alternatives from the point of view of someone  
> who is trying to built a platform for sale, me. The platform is to  
> allow many users to host and run their trading strategies  
> unattended, somewhere on a server co-located near the exchange. The  
> platform has to have a web-based user and admin interface. I want to  
> be able to compile strategies to machine code for fast execution. I  
> also want some kind of typing to catch my errors and give me an  
> additional layer of assurance.
In my experience, typing really doesn't catch errors anymore.  That  
said, dialyzer is wonderful.

> I can write the whole thing in Lisp. Lisp has a built-in compiler  
> and good web servers. I can take an HTTP POST-ed strategy and  
> compile it, making sure it will run fast. I have a free license to a  
> commercial Lisp but it requires royalties. I don't think I want to  
> go there and I don't want to spend money on a non-royalty commercial  
> Lisp either. Neither Lisp will give me fault tolerance, scalability  
> or typing.
Agreed.

> OCaml. I like the static typing, speed and ease of use (for me). I  
> had the most pleasant experience of all writing and debugging my  
> compiler in OCaml. OCaml does not have a built-in compiler, though.  
> I will need to schedule jobs to compile strategies. The OCaml web  
> server (Ocsigen) is not exactly proven and OCaml does not give me  
> scalability of fault tolerance.
Okay.  Haskell might be another option.  If you're going typed, it's  
the best, but it's probably suboptimal due to lack of libraries.

> C++. I like the static typing and Boost Spirit v2.1 seems like the  
> ticket for writing the compiler. Performance can truly be squeezed  
> here. I could build my engine into a web server (nginx?) and use  
> LLVM or NanoJIT to compile strategies. I will surely spend a hell of  
> a long time developing compared to other solutions. C++ does not  
> give me scalability but I could use ZeroMQ or something similar.  
> Fault tolerance will need to be ensured by making each trading  
> strategy into a separate process.
Okay.  Also, avoid templates and try compiling for size.  In cache- 
sensitive environments, I've found that -Os gets me better performance  
that -O99 just because of the smaller code size.
>
> Erlang. There's a layer of typing with dialyzer. The web servers are  
> good, there's scalability and fault tolerance. Building a compiler  
> should be straightforward with leex and yecc. I can take HTTP POST- 
> ed strategies and compile them. There's always HiPE for that  
> ultimate performance boost. Performance may not be top-notch in the  
> end but development should be fast and debugging straightforward.
In particular, debugging of concurrent code can be very nice.  http://souja.net/2009/04/making-sense-of-erlangs-event-tracer.html

> I see migrating to C++ as need arises, chunk by chunk. Erlang may  
> end up as the message bus or I may swap it out entirely. This won't  
> happen for a while or may never happen at all. Erlang just seems  
> like the lesser of all evils or, perhaps, the golden middle.
That makes sense.  It sounds like the selling points are  
debuggableness, fault-tolerance, scalability, and concurrency.  That  
all makes sense.  Good luck on the latency front, though, as each of  
those traits increases latency.


-- 
Jayson Vantuyl
kagato@REDACTED