[erlang-questions] benchmarks game harsh criticism

[Isaac Gouy]

--- David Hopwood <david.hopwood@REDACTED> wrote:

-snip- 
> Proportion of language implementations for which each benchmark
> included in the default weighting takes less than 10, 30, and 60
> seconds CPU time...

I don't wish to intrude but you haven't said whether the data you
present is from the Debian AMD measurements or the Gentoo Intel
measurements, the Extra languages ...


-snip-
> naïve Sieve of Eratosthenes ...
> Isn't it more likely that there is some source of
> constant overhead that is causing the variation between languages to
> be compressed?

I suspect that's approaching top speed for a naïve algorithm on this
machine :-)


 
> Another basic mistake is that there is no indication of the variation
> in timing between benchmark runs. At least, not without digging a bit
> deeper: the excluded "Haskell GHC #4" result for N=9 on nsieve is
> 1.12 s, but in the full results for nsieve-bits, the result for N=9
on
> exactly the same program run by the same language implementation is
> 0.80 s.

Are you sure the basic mistake is not yours?

Gentoo : Intel® Pentium® 4 

nsieve,ghc,4,9, 1.124,

nsievebits,ghc,4,9, 1.112




      ____________________________________________________________________________________
Get easy, one-click access to your favorites. 
Make Yahoo! your homepage.
http://www.yahoo.com/r/hs