[erlang-questions] benchmarks game harsh criticism
Thu Nov 29 07:31:46 CET 2007
--- David Hopwood <david.hopwood@REDACTED> wrote:
> Proportion of language implementations for which each benchmark
> included in the default weighting takes less than 10, 30, and 60
> seconds CPU time...
I don't wish to intrude but you haven't said whether the data you
present is from the Debian AMD measurements or the Gentoo Intel
measurements, the Extra languages ...
> naïve Sieve of Eratosthenes ...
> Isn't it more likely that there is some source of
> constant overhead that is causing the variation between languages to
> be compressed?
I suspect that's approaching top speed for a naïve algorithm on this
> Another basic mistake is that there is no indication of the variation
> in timing between benchmark runs. At least, not without digging a bit
> deeper: the excluded "Haskell GHC #4" result for N=9 on nsieve is
> 1.12 s, but in the full results for nsieve-bits, the result for N=9
> exactly the same program run by the same language implementation is
> 0.80 s.
Are you sure the basic mistake is not yours?
Gentoo : Intel® Pentium® 4
Get easy, one-click access to your favorites.
Make Yahoo! your homepage.
More information about the erlang-questions