[erlang-questions] Accounting Engine in Erlang

[Jesper Louis Andersen]

On Mon, Jun 14, 2010 at 5:08 PM, amiruddin nagri <amir.nagri@REDACTED> wrote:

> Is the issue of consistency not solvable using erlang, or if we
> implement consistency using erlang we won't be able use the
> performance and scaling feature of erlang to maximum.

Consistency is certainly to be had while getting good performance and
scaling at the same time. How you realize it is up to you however. You
should expect to spend some time in the optimization department if you
want the application to scale. Perhaps the kool-aid says otherwise,
but in reality you can't circumvent spending a little bit of time with
the profiler and plan ahead.

> Also, if you have any suggestions of the choice of programming
> language, we have been looking into other functional languages like
> Haskell and Clojure. But we have not dig deep on the performance
> aspects of these languages, if someone can shed a light on the pros-
> cons of these languages, it will help us very much to come to a
> decision.

I can only speak for Erlang vs. Haskell. The shameless plug is:

http://jlouisramblings.blogspot.com/2010/04/haskell-vs-erlang-for-bittorent-clients.html

In Haskell, the perhaps biggest problem will be laziness. For a
long-running daemon process it is very easy to end up with a
space-leak due to unevaluated lazy thunks. You will have to constantly
look at heap profiles to be sure you are not reintroducing a leak when
making changes. Expect to spend some time learning how to strictness
annotate your programs if you choose Haskell. On the other hand,
Haskell's type system will eradicate most bugs early on if your
exploit the type system rather than seeing it as a straitjacket.
Nothing, save Agda, Epigram or Coq, have a more powerful type system
out there. And you can expect a compiler like GHC to produce fairly
fast code (provided you know what you are doing).

Bottom line of etorrent vs. Combinatorrent is that I have spent far
less time optimizing etorrent compared to combinatorrent. But they
perform around the same in CPU usage. The Erlang client use more
memory, but only a constant factor of some 2-3 times, which is far
from alarming (we are still beating the python/C(++) and java clients
:). But it took some time with Erlang before I figured out the
tradeoffs between mnesia, ETS tables and keeping a large process heap
with a dict/gb_tree etc. I have come to call this knowledge the
"performance profile" of a language. And, not too surprisingly, it
turns out that these are very different from language to language (and
implementation to implementation). You should expect to spend some
time with a new language before you get the performance profile under
your skin.

Finally, you should not put too much into synthetic benchmarks. Anil
Madhavapeddy (http://http://anil.recoil.org/) found, in his Phd
thesis, that writing an SSH client in Ocaml made it much faster than
the C client. Yet, C beats Ocaml on most synthetic benchmarks. The
argument carries over to, say, Erlang: You can expect your program to
reach correctness faster and can thus spend more time on boosting raw
performance of the program.

There is another important property of Erlang: inspection. For a
running Erlang program you can dynamically probe, inspect and trace
its operation. In real life, this is a very powerful tool in the
toolbox. Numerous times, I have been able to figure out and fix
trouble by inspecting the running application. I'm happy to pay quite
much raw performance on the prayer altar in order to have this
ability.

And then there is distribution over multiple nodes, where Erlang is
basically running circles around anything else I have seen...,

-- 
J.