[erlang-questions] Benchmarks

[Henning Diedrich]

Thanks a lot for elaborating -- !

I am still looking for a briefer expression of the posted answers. Which 
can't replace the understanding of the right application domain as you 
are pointing out, but may simply serve different purpose.

Maybe there is a link somewhere I don't find and you can point me to it. 
Maybe not and you could help correct and complete the following? It is 
guessed and interpolated, trying to describe what I am looking for. And 
it needs not be exact, it's a matter of magnitudes - and brevity and 
simplicity of talking points in business situations.

                                               Erlang   Java    C++   
C   PHP   Ruby   Python  Stackless  Haskell   Go   FORTRAN

Max processes/ Threads on an average* machine   100,000  10,000
Time to create a process/thread in us           ~10      ~100
Footprint of process/thread in KB                 1        8
Expressiveness in LOC per LOC in C                4                   1 
              
Performance in Ring Passing Benchmark (1/sec)    
FLOP/s with built in float format
Best known uptime nines of production products    9
Biggest known commercial systems in LOC          n*10^7
Biggest known commercial systems in team members n*10^2
Killable by loose pointers                       no
Killable by memory access dead locks             no
Arguable tendency to run at first compile        yo                   lo
Transparent use of multicore architecture       yes      no     no    no
Built in arbitrary precision math               yes ?
Easy to use built in string handling             no
+ ...!

*whatever that is, obviously only important for comparison of magnitudes 
and so eventually mostly cancelled out, and let's even use a multicore 
here playing to the Erlang VM's strengths, where that can make a 
difference ...

... and clearly designate the weaker sides, too, if you will.

Sure, to answer the question for fitness of purpose must rely on 
clarifying what the purpose is, but benchmarks may still govern what 
get's on the short list in the first place. Maybe particularily Erlang's 
strength's defy being measured in numbers? I can't  imagine that AT ALL. 
All the parallel stuff plays into its hands. If it's number crunching 
that makes Erlang look bad, then the more suitable benchmark tests are 
of interest.

To get out of defensive mode vs. other languages, the matter of the fact 
is that I'd be on the lookout for something as brief as, and projecting 
as much science as, a factor or percentile. I am saying projecting 
science. But of course, not made up, either.

Shifting the buts and ifs to the part of the detractors then, 
preferrably. Who might then go ahead, if they prefer, to point out how 
the test was skewed, allegedly in favor of Erlang. Someone will always 
rightly have to explain something.

There are findings along the line of "1/4 of code compared to C", even 
if (or especially if!) not arrived at by a simple formula, but by a 
hands on experiment like Richard Jones (, Esq.)

"Rewriting Playdar: C++ to Erlang, massive savings"
http://www.metabrew.com/article/rewriting-playdar-c-to-erlang-massive-savings

Certainly any "fact", if benchmark result or anecdotical wisdom, will be 
debatable.

But are there more like

http://pseudogreen.org/blog/erlang_vs_stackless_vs_multitask.html
http://www.joeandmotorboat.com/2009/01/03/nginx-vs-yaws-vs-mochiweb-web-server-performance-deathmatch-part-2/
http://www.sics.se/~joe/apachevsyaws.html

and are the individual, published answers to Joe Armstrong's ring 
benchmark tasks coalesking into a clear picture? What is Erlang's sure 
fire, small talk, chocolate benchmark?

 From the latter link you could come away saying that an Erlang server 
could do 20 times more sessions than Apache. Did this hold up? I thought 
it was discussed heatedly but is that 20x factor regarded as valid 
statement in the Erlang community today?

Are there current benchmarks along the lines of

"Performance Measurements of Threads in Java and Processes in Erlang"
http://www.sics.se/~joe/ericsson/du98024.html

which latter would allow to state that Erlang can create processes 
around a hundred times faster than Java creates new threads and can use 
ten times more (only??) of them etc.

The notion that for an actor model language, a comparison of processes 
and objects might make more sense than processes and threads sometimes, 
is then part of the ifs and buts. And rightly so. But anyone listening 
to anyone evangelizing will add a grain of salt anyway. There are 
qualifications with every number out there. But what are the Erlang ones?

I refrained from studying Armstrong's thesis and the Motorola study, so 
I am not aware what I may be missing there.

Thanks,
Henning




Ulf Wiger wrote: