[erlang-questions] couchdb performace 10x: using NIF for file io

Paul Davis paul.joseph.davis@REDACTED
Mon Oct 25 16:25:11 CEST 2010

On Sun, Oct 24, 2010 at 4:59 PM, Joel Reymont <joelr1@REDACTED> wrote:
> On Oct 24, 2010, at 7:16 PM, Kenneth Lundin wrote:
>> I wonder how responsive the system is to other events when running the
>> benchmark.
> The benchmark simulates several hundred clients hitting a (mochiweb) web server to read and write couchdb (json) documents. The system seems to stay -highly- responsive, 10x so compared to the same system not using NIFs at all.
> If low write response time is taken as a measure of system responsiveness then the first graph shows that the responsiveness of the system has increased dramatically. The write response here is the take taken to process a web request to write a couch document.
> ---
> http://twitter.com/wagerlabs
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED


I think the issue here is the way that we sequester file io to a
single Erlang process. In your benchmark you'd basically be
sacrificing an Erlang scheduler thread to speed up the synchronous
writes while the other schedulers are freely available to handle
read/write requests.

I'm not sure if Mikeal has written a test for Relaximation that runs a
similar test that hits multiple databases. I think the way I'd try and
show the issue would be to have a large number of clients attempting
to write to their own database.

Also, a trivial way to prove to your self that the NIF's are indeed
called in the same scheduler threads is to create a NIF that has a
function that just does "while(1){}". If you call that function in
more processes than you have schedulers the VM will halt.

Paul Davis

More information about the erlang-questions mailing list