[erlang-questions] basho_bench, was Re: [ANN] Silly benchmarking

[Scott Lystig Fritchie]

Michael Truog <mjtruog@REDACTED> wrote:

mt> The other option is trying to use basho_bench here
mt> https://github.com/basho/basho_bench, if you are testing key/value
mt> storage.

Actually, you can use it to measure whatever you want, if you write a
callback module with a new/1 and run/4 function.  Tracking throughput is
always good, but also tracking min, mean, median, 95th, 99th, 99_9th,
and maximum latency stats plus error rates is An Even BetterThing(tm).
Nice R and Gnuplot graphs is also very convenient.

For an example graph, see
http://www.snookles.com/scotttmp/basho_bench_null_summary.png ... It
used the bundled examples/null_test.config config, which tests
approximately nothing but basho_bench's internal overhead.  :-)

That graph was generated on an 8 HT core MacBook Pro (with a fair amount
of other unrelated stuff running in parallel, CPU frequency changing due
to heat, etc etc.), using today's 'master' branch:

    git clone git://github.com/basho/basho_bench.git
    cd basho_bench
    make
    ./basho_bench examples/null_test.config
    make results

... and the graph is at tests/current/summary.png.

-Scott