[erlang-questions] Who is interested in an Erlang Performance Improvement Book?

[Jesper Louis Andersen]

On Fri, Aug 14, 2015 at 2:28 PM, Loïc Hoguin <essen@REDACTED> wrote:

> The time spent to run X is the worst kind of benchmark value you can rely
> on though.


This would be a worry of me as well. The Go benchmark runner is better than
nothing, but it is highly naive. One, it doesn't try to figure out garbage
collector interference. Two, it doesn't cook benchmarks until they reach a
stable state. Three, it doesn't carry out the proper statistics. A simple
bootstrapping analysis of the average could tell you if the average is
stable or is perturbed by some internal thing in the system.

The problem may not even be tied to the Erlang node. When you are measuring
highly sensitive benchmarks at the nanosecond scale, a simple interrupt may
be enough to mess up the benchmark. And you can't in general control for
this in a complex environment.


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150814/19d2a6ab/attachment.htm>