[erlang-questions] couchdb performace 10x: using NIF for file io

[Paul Davis]

On Wed, Oct 27, 2010 at 3:46 AM, Edmond Begumisa
<ebegumisa@REDACTED> wrote:
> Hi,
>
> I hope the Couch team isn't planning on doing this by default -- something
> about it makes me nervous...
>
> When CouchDB is on it's own, it might not be alarming/noticeable, but I'm
> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e. where
> Couch is just one of many OTP apps running in the *SAME* VM -- I have a few
> hacks for avoiding socket communication.) I too worry about the potential
> for NIF-endowed couch io disturbing the balance of Erlang's scheduling.
>
> It would be good to see similar benchmarking with the VM concurrently doing
> things other than handling couch-related requests (which are implicitly
> synchronised in your case.)
>
> - Edmond -
>

Edmond,

You don't have to worry about us changing away from the default
configuration without making sure that different types of loads are
either unaffected for similarly improved. I've already had to address
this specific issue when integrating Emonk in an experimental branch
so it was the first thing I noticed about the file descriptor patch.

Out of curiosity, do you have another Erlang app that would be a good
candidate for using to test that other parts of the VM remain
responsive. I was going to suggest various parts of CouchDB that don't
touch IO as a smoke screen, but an app doing something real that we
can measure in and out of couch and with and without the new file io
would be a good help.

Paul Davis

> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont <joelr1@REDACTED> wrote:
>
>>
>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>>
>>> I wonder how responsive the system is to other events when running the
>>> benchmark.
>>
>> The benchmark simulates several hundred clients hitting a (mochiweb) web
>> server to read and write couchdb (json) documents. The system seems to stay
>> -highly- responsive, 10x so compared to the same system not using NIFs at
>> all.
>>
>> If low write response time is taken as a measure of system responsiveness
>> then the first graph shows that the responsiveness of the system has
>> increased dramatically. The write response here is the take taken to process
>> a web request to write a couch document.
>>
>> ---
>> http://twitter.com/wagerlabs
>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>
>
> --
> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>