[erlang-questions] zlib driver in erlang seems very slow

[Caragea Silviu]

Hello guys,

While investigating some benchmarks with zlib on erlang I suspected that
the elrang driver might be too slow.
Initially I wrote a small c++ app linking with zlib doing almost the same
stuff like my erlang testing script and I found a big gap in performance
between them.
Initial thought was that the difference might come from the overhead added
by erlang itself. (converting from elrang terms into c++ data and vice
versa)

Then I started creating my own nif library for zlib and I found that
comparing with erlang driver it's around 30 % faster for doing the same job.
After other investigations on zlib I found some interesting articles
regarding intel and cloudflare patches and also zlibng that performs much
better than
the original zlib. After I linked my NIF with this zlib libraries I got a
gain of around 69 % compared with erlang driver.

The testing script is here:
https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl

Basically reads a file line by line and compress it N times.

For example:

    benchmark:run(ezlib,"file path here", 200, 6, 10, 1).
    benchmark:run(erlang,"file path here", 200, 6, 10, 1).

This is compressing the file 200 times with a compression level of 6 window
size 10 and memory level 1. First one using my library second one using
erlang driver.

Some benchmark results you can see on the https://github.com/silviucpp/ezlib

I can understand that the erlang driver api is generic and allows you to
customize a lot if things and mine it's not and it's built
around my need to use it with ejabberd but still I'm almost sure if I
change the interface the performance difference will be the same.

Maybe my benchmark is not realistic or I'm missing something.
Feedback it's welcome.

Silviu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160309/586e2029/attachment.htm>