[erlang-questions] couchdb performace 10x: using NIF for file io

[Edmond Begumisa]

You could try embedding yaws serving both static and dynamic content  
(completely unrelated to couch's requests.) You could then also use  
hovercraft for couch io -- I imagine this would eliminate yaws + couch's  
mochiweb sharing the socket driver thereby giving a better picture of how  
NIF io affects the yaws side.

- Edmond -


On Thu, 28 Oct 2010 01:19:18 +1100, Paul Davis  
<paul.joseph.davis@REDACTED> wrote:

> On Wed, Oct 27, 2010 at 3:46 AM, Edmond Begumisa
> <ebegumisa@REDACTED> wrote:
>> Hi,
>>
>> I hope the Couch team isn't planning on doing this by default --  
>> something
>> about it makes me nervous...
>>
>> When CouchDB is on it's own, it might not be alarming/noticeable, but  
>> I'm
>> using CouchDB "embedded" in a wider Erlang/OTP application stack (i.e.  
>> where
>> Couch is just one of many OTP apps running in the *SAME* VM -- I have a  
>> few
>> hacks for avoiding socket communication.) I too worry about the  
>> potential
>> for NIF-endowed couch io disturbing the balance of Erlang's scheduling.
>>
>> It would be good to see similar benchmarking with the VM concurrently  
>> doing
>> things other than handling couch-related requests (which are implicitly
>> synchronised in your case.)
>>
>> - Edmond -
>>
>
> Edmond,
>
> You don't have to worry about us changing away from the default
> configuration without making sure that different types of loads are
> either unaffected for similarly improved. I've already had to address
> this specific issue when integrating Emonk in an experimental branch
> so it was the first thing I noticed about the file descriptor patch.
>
> Out of curiosity, do you have another Erlang app that would be a good
> candidate for using to test that other parts of the VM remain
> responsive. I was going to suggest various parts of CouchDB that don't
> touch IO as a smoke screen, but an app doing something real that we
> can measure in and out of couch and with and without the new file io
> would be a good help.
>
> Paul Davis
>
>> On Mon, 25 Oct 2010 07:59:11 +1100, Joel Reymont <joelr1@REDACTED>  
>> wrote:
>>
>>>
>>> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>>>
>>>> I wonder how responsive the system is to other events when running the
>>>> benchmark.
>>>
>>> The benchmark simulates several hundred clients hitting a (mochiweb)  
>>> web
>>> server to read and write couchdb (json) documents. The system seems to  
>>> stay
>>> -highly- responsive, 10x so compared to the same system not using NIFs  
>>> at
>>> all.
>>>
>>> If low write response time is taken as a measure of system  
>>> responsiveness
>>> then the first graph shows that the responsiveness of the system has
>>> increased dramatically. The write response here is the take taken to  
>>> process
>>> a web request to write a couch document.
>>>
>>> ---
>>> http://twitter.com/wagerlabs
>>>
>>>
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>
>>
>>
>> --
>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/