[erlang-questions] Who is interested in an Erlang Performance Improvement Book?

[Alexander Shorin]

On Fri, Aug 14, 2015 at 3:28 PM, Loïc Hoguin <essen@REDACTED> wrote:
> On 08/14/2015 02:12 PM, Alexander Shorin wrote:
>>
>> On Fri, Aug 14, 2015 at 2:58 PM, Loïc Hoguin <essen@REDACTED> wrote:
>>>
>>>
>>> In other words: we have fantastic test tools to ensure we don't add new
>>> bugs
>>> with each commit; what would you do to make sure you don't kill
>>> performance
>>> with each commit?
>>>
>>> A good answer to that alone would be worth a thousand books on
>>> performance.
>>
>>
>> Golang had used special benchmark test suite which ran on each commit
>> against various configurations[1] and it was quite trivial to detect
>> which commit caused performance and for what case, but suddenly it
>> doesn't works anymore. Some bits web archive still remember[2] about
>> how it looked like.
>>
>> But in general, solution of this problem is the same: make benchmarks
>> as yet another test suite and track how long it being executed, not
>> just ok/error status. At some point turning integration tests into
>> benchmarks might be good idea.
>
>
> The time spent to run X is the worst kind of benchmark value you can rely on
> though.
>
> In Erlang for example you may replace some code with a NIF (or even a BIF),
> effectively making these benchmarks faster, only to find out later that it
> makes the whole system slower when you have 100k processes using it
> concurrently.
>
> Or perhaps a piece of code will run faster, but with double memory usage. Or
> the latency suffers. Etc.
>
> One thing for sure is that you need a benchmark test suite, but how to set
> it up to make sure the system as a whole benefits from all the changes is a
> guide I've never seen written.

Good points. But if we move from "speed of code" to "behaviour of
code", then for such comprehensive measurements  we eventually involve
monitoring systems into the process which also need to register "side
effects" of the underlying system to make results more or less clear
to understand. This all turns into good infrastructure setup task and
devops domain.

--
,,,^..^,,,