[erlang-questions] zlib driver in erlang seems very slow
Wed Mar 9 18:25:54 CET 2016
Sadly Concurix has "pivoted" to instrumenting nodejs apps.
On 9 March 2016 at 12:39, Loïc Hoguin <essen@REDACTED> wrote:
> Erlang/zlib using only one core fits the discussion I had about it. Not sure
> about the OOM crash though.
> By the way I don't know if there's anything out there about it but Joe
> mentioned a lock in zlib here:
> https://joearms.github.io/2013/03/28/solving-the-wrong-problem.html - Little
> details, and I don't think they open sourced their zlib implementation
> On 03/09/2016 01:28 PM, Caragea Silviu wrote:
>> Yes so your suggestion to test concurrently was very good. I updated the
>> testing script and running it on my machine with 1000 concurrent
>> processes It's crashing the VM using erlang driver . I suspect because
>> out of memory. My machine has only 16 GB and I see at some point 48 GB
>> in the beam process.
>> Anyway before crashing all the time I see cpu consumption being 100 %
>> maximum on beam process.
>> The same test using my library complete fine. also the cpu I see cpu
>> staying at 500 % - 600 % and sometimes 700% which means it's using
>> almost all the cores I have. Memory wise is not using more than : 12 GB
>> Something is strange. You can review my testing program maybe I'm wrong
>> On Wed, Mar 9, 2016 at 1:15 PM, Caragea Silviu <silviu.cpp@REDACTED
>> <mailto:silviu.cpp@REDACTED>> wrote:
>> Hello Loic,
>> I didn't tested concurrently but I will do it shouldn't be very
>> difficult to change the testing script. Even if from what I know
>> zlib doesn't have any mutex or something to lock the access to
>> resources inside.
>> Regarding inflateChunk I might add it, right now its only an
>> interface that makes integration in the projects I'm using faster.
>> On Wed, Mar 9, 2016 at 1:08 PM, Loïc Hoguin <essen@REDACTED
>> <mailto:essen@REDACTED>> wrote:
>> On 03/09/2016 11:54 AM, Caragea Silviu wrote:
>> Some benchmark results you can see on the
>> I can understand that the erlang driver api is generic and
>> allows you to
>> customize a lot if things and mine it's not and it's built
>> around my need to use it with ejabberd but still I'm almost
>> sure if I
>> change the interface the performance difference will be the
>> Maybe my benchmark is not realistic or I'm missing something.
>> Feedback it's welcome.
>> I have recently come to realize the same thing as you. A few
>> discussions I had with users pointed out that the performance of
>> the Erlang zlib might be even worse when used concurrently. Have
>> you done some measurements on that?
>> Also you will probably need an inflateChunk equivalent if you
>> are doing anything public facing. I don't think it changes much
>> performance-wise though.
>> Loïc Hoguin
>> Author of The Erlanger Playbook,
>> A book about software development using Erlang
> Loïc Hoguin
> Author of The Erlanger Playbook,
> A book about software development using Erlang
> erlang-questions mailing list
More information about the erlang-questions