[erlang-questions] trading systems
Thu Oct 1 21:43:56 CEST 2009
> True. I think I'll still start by experimenting with mmap-ed
> binaries, or at least a binaries-based approach. These can always be
> mmap-ed later.
Erlang assumes that binaries don't change. Note that it never
modifies them, only makes new ones. Given that the compiler optimizes
for this (or should), I would be careful about modifying binaries
after they make it to Erlang.
> Yes, I may try porting HiPE to x86-64 when I have time to spare.
If you're very speed conscious, I would recommend running in 32-bit
mode. With everything I've done, it seems faster to page a GB of data
in/out of memory than it does to deal with the larger pointers--
especially in something pointer-heavy (like a dynamic language).
> Let's consider a few alternatives from the point of view of someone
> who is trying to built a platform for sale, me. The platform is to
> allow many users to host and run their trading strategies
> unattended, somewhere on a server co-located near the exchange. The
> platform has to have a web-based user and admin interface. I want to
> be able to compile strategies to machine code for fast execution. I
> also want some kind of typing to catch my errors and give me an
> additional layer of assurance.
In my experience, typing really doesn't catch errors anymore. That
said, dialyzer is wonderful.
> I can write the whole thing in Lisp. Lisp has a built-in compiler
> and good web servers. I can take an HTTP POST-ed strategy and
> compile it, making sure it will run fast. I have a free license to a
> commercial Lisp but it requires royalties. I don't think I want to
> go there and I don't want to spend money on a non-royalty commercial
> Lisp either. Neither Lisp will give me fault tolerance, scalability
> or typing.
> OCaml. I like the static typing, speed and ease of use (for me). I
> had the most pleasant experience of all writing and debugging my
> compiler in OCaml. OCaml does not have a built-in compiler, though.
> I will need to schedule jobs to compile strategies. The OCaml web
> server (Ocsigen) is not exactly proven and OCaml does not give me
> scalability of fault tolerance.
Okay. Haskell might be another option. If you're going typed, it's
the best, but it's probably suboptimal due to lack of libraries.
> C++. I like the static typing and Boost Spirit v2.1 seems like the
> ticket for writing the compiler. Performance can truly be squeezed
> here. I could build my engine into a web server (nginx?) and use
> LLVM or NanoJIT to compile strategies. I will surely spend a hell of
> a long time developing compared to other solutions. C++ does not
> give me scalability but I could use ZeroMQ or something similar.
> Fault tolerance will need to be ensured by making each trading
> strategy into a separate process.
Okay. Also, avoid templates and try compiling for size. In cache-
sensitive environments, I've found that -Os gets me better performance
that -O99 just because of the smaller code size.
> Erlang. There's a layer of typing with dialyzer. The web servers are
> good, there's scalability and fault tolerance. Building a compiler
> should be straightforward with leex and yecc. I can take HTTP POST-
> ed strategies and compile them. There's always HiPE for that
> ultimate performance boost. Performance may not be top-notch in the
> end but development should be fast and debugging straightforward.
In particular, debugging of concurrent code can be very nice. http://souja.net/2009/04/making-sense-of-erlangs-event-tracer.html
> I see migrating to C++ as need arises, chunk by chunk. Erlang may
> end up as the message bus or I may swap it out entirely. This won't
> happen for a while or may never happen at all. Erlang just seems
> like the lesser of all evils or, perhaps, the golden middle.
That makes sense. It sounds like the selling points are
debuggableness, fault-tolerance, scalability, and concurrency. That
all makes sense. Good luck on the latency front, though, as each of
those traits increases latency.
More information about the erlang-questions