Computer Language Shootout - concurrency

[Marthin Laubscher]

[Marthin Laubscher]
In stead of benchmarking execution time, what about benchmarks for:
 Time to market
 Reliability/
 Fault Tolerance
 Scalability
 Predictability of behaviour
 Predictability of performance
 Processing Distribution
   (Design, dev & test cost, deployment and network cost)
 Release Management
 Programmer Productivity
 Test Productivity

If we need to base decisions on benchmarks, such measurements would make far
more sense to me than CPU cycles.

Any team or individual that base development decisions on such measurements
as the shootout benchmarks put forward should IMHO be left alone to stew in
its own juices.

> -----Original Message-----
> From: owner-erlang-questions@REDACTED [mailto:owner-erlang-
> questions@REDACTED] On Behalf Of Peter-Henry Mander
> Sent: 06 December 2005 11:19
> To: Erlang Questions
> Subject: Re: Computer Language Shootout - concurrency
> 
<snip synopsis="fairness of benchmarks"/>
> <rant>
> 
> Why is there not another concurrency test with 5000 threads and more
> that demonstrate the absolute dominance of erlang? How can the
> benchmarks be useful unless they highlight those languages with unique
> strengths such as concurrency in erlang? Are there other benchmarks that
> have been 'politically-corrected' or 'dumbed-down' to be overly
> inclusive?
> 
> </rant>
> 
> I'm trying to find time to rectify what erlang code I can, but it's a
> very low priority. But until _all_ the benchmarks are corrected and
> peer-reviewed the ranking is utterly pointless. If anyone cares (and
> obviously some of us do) we should extract our proverbial digit and
> 'fix' the results by submitting better code, and persuade the benchmark
> designers to add better and wider-ranging concurrency tests, since
> concurrency is becomming a hot topic these days with multicore and cell
> processors.
>
<snip synopsis="how we should improve the code used in the benchmarks"/> 
> 

[Marthin Laubscher] Again, suppose we did all that, what would it prove? Why
volunteer to be dragged down to such petty comparisons? :-)