[erlang-questions] zlib driver in erlang seems very slow

[Caragea Silviu]

Yes because I'm compressing small chunks that anyway takes less than 1-2 ms

Silviu

On Wed, Mar 9, 2016 at 1:02 PM, Sergej Jurečko <sergej.jurecko@REDACTED>
wrote:

> If you care about compression speed zlib is a terrible choice anyway. Also
> in your NIF you seem to be doing the entire compression on a scheduler
> thread all in one go. This is very bad. NIF calls should not take more than
> ~1ms.
>
> Sergej
>
> On 09 Mar 2016, at 11:54, Caragea Silviu <silviu.cpp@REDACTED> wrote:
>
> Hello guys,
>
> While investigating some benchmarks with zlib on erlang I suspected that
> the elrang driver might be too slow.
> Initially I wrote a small c++ app linking with zlib doing almost the same
> stuff like my erlang testing script and I found a big gap in performance
> between them.
> Initial thought was that the difference might come from the overhead added
> by erlang itself. (converting from elrang terms into c++ data and vice
> versa)
>
> Then I started creating my own nif library for zlib and I found that
> comparing with erlang driver it's around 30 % faster for doing the same job.
> After other investigations on zlib I found some interesting articles
> regarding intel and cloudflare patches and also zlibng that performs much
> better than
> the original zlib. After I linked my NIF with this zlib libraries I got a
> gain of around 69 % compared with erlang driver.
>
> The testing script is here:
> https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl
>
> Basically reads a file line by line and compress it N times.
>
> For example:
>
>     benchmark:run(ezlib,"file path here", 200, 6, 10, 1).
>     benchmark:run(erlang,"file path here", 200, 6, 10, 1).
>
> This is compressing the file 200 times with a compression level of 6
> window size 10 and memory level 1. First one using my library second one
> using erlang driver.
>
> Some benchmark results you can see on the
> https://github.com/silviucpp/ezlib
>
> I can understand that the erlang driver api is generic and allows you to
> customize a lot if things and mine it's not and it's built
> around my need to use it with ejabberd but still I'm almost sure if I
> change the interface the performance difference will be the same.
>
> Maybe my benchmark is not realistic or I'm missing something.
> Feedback it's welcome.
>
> Silviu
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160309/41e60b63/attachment.htm>