<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">If you care about compression speed zlib is a terrible choice anyway. Also in your NIF you seem to be doing the entire compression on a scheduler thread all in one go. This is very bad. NIF calls should not take more than ~1ms.</div><div class=""><br class=""></div><div class="">Sergej</div><div class=""><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 09 Mar 2016, at 11:54, Caragea Silviu <<a href="mailto:silviu.cpp@gmail.com" class="">silviu.cpp@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hello guys,<br class=""><br class="">While investigating some benchmarks with zlib on erlang I suspected that the elrang driver might be too slow. <br class="">Initially I wrote a small c++ app linking with zlib doing almost the same stuff like my erlang testing script and I found a big gap in performance between them.<br class="">Initial thought was that the difference might come from the overhead added by erlang itself. (converting from elrang terms into c++ data and vice versa)<br class=""><br class="">Then I started creating my own nif library for zlib and I found that comparing with erlang driver it's around 30 % faster for doing the same job.<br class="">After other investigations on zlib I found some interesting articles regarding intel and cloudflare patches and also zlibng that performs much better than <br class="">the original zlib. After I linked my NIF with this zlib libraries I got a gain of around 69 % compared with erlang driver. <br class=""><br class="">The testing script is here: <a href="https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl" class="">https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl</a><br class=""><br class="">Basically reads a file line by line and compress it N times.<br class=""><br class="">For example: <br class=""><br class=""> benchmark:run(ezlib,"file path here", 200, 6, 10, 1). <br class=""> benchmark:run(erlang,"file path here", 200, 6, 10, 1).<br class=""><br class="">This is compressing the file 200 times with a compression level of 6 window size 10 and memory level 1. First one using my library second one using erlang driver.<br class=""><br class="">Some benchmark results you can see on the <a href="https://github.com/silviucpp/ezlib" class="">https://github.com/silviucpp/ezlib</a><br class=""><br class="">I can understand that the erlang driver api is generic and allows you to customize a lot if things and mine it's not and it's built <br class="">around my need to use it with ejabberd but still I'm almost sure if I change the interface the performance difference will be the same. <br class=""><br class="">Maybe my benchmark is not realistic or I'm missing something. <br class="">Feedback it's welcome.<br class=""><br class="">Silviu<br class=""> <br class=""></div>
_______________________________________________<br class="">erlang-questions mailing list<br class=""><a href="mailto:erlang-questions@erlang.org" class="">erlang-questions@erlang.org</a><br class="">http://erlang.org/mailman/listinfo/erlang-questions<br class=""></div></blockquote></div><br class=""></div></div></body></html>