[erlang-questions] benchmarks game harsh criticism
Tue Dec 4 17:57:18 CET 2007
--- Bengt Kleberg <bengt.kleberg@REDACTED> wrote:
> so, you think that the 3 quotes from "Timing Trials" does not
> a certain method to show additional insights, instead they describe a
> certain method to show additional insights. that is ok with me. i can
> change from recomend to describe, without losing track of the target.
> moreover, i will assume that you mention "C runtimes appears
> horizontal" [for this test], as one example when "Timing Trials"
> "anomalous behavior that deserves further attention." that is no
> with me. (if you mean that this is the one and only thing ever to
> deserve more investigation, i would like an explanation on how you
> managed to arrive at that idea.)
> anyway then, we seem to agree on the following: "Timing Trials"
> that it is a good idea to have sufficiently many measuring points
> benchmarking to be able to spot anomalous behaviour. i think the
> shootout does not do this and that it would be a good thing if it
> you do not want the shootout do this, for reasons never explained.
Do you agree that the benchmarks game has a range of input values ~10x
to ~100x, in comparison to mostly < 10x in "Timing Trials"?
Do you agree that the benchmarks game range of input values /has/ been
used to spot anomalous behaviour - binary-trees memory usage not
increasing for some programs?
Be a better sports nut! Let your teams follow you
with Yahoo Mobile. Try it now. http://mobile.yahoo.com/sports;_ylt=At9_qDKvtAbMuh1G1SQtBI7ntAcJ
More information about the erlang-questions