[erlang-questions] couchdb performace 10x: using NIF for file io

[Kenneth Lundin]

Hi,

Have you tried the very same benchmark with just using the 'raw'
option when opening the file.
You should then get about the same efficiency as you get with NIFs
with the difference that you
can use asynch threads as well (i.e . the +A option will be
effective). Without the +A you will get the same
setup as you get with your NIF example.
Another benefit with that is that you don't need to write any C-code.

If the NIF example still is much faster (which I doubt) please let me know.

Without use of the raw option all file operations is done via an extra
Erlang process which explains the difference in speed.

/Kenneth Erlang/OTP, Ericsson

On Sun, Oct 24, 2010 at 3:56 PM, Joel Reymont <joelr1@REDACTED> wrote:
> Simply switching to NIFs for file IO seems to have improved CouchDB write performance more than ten-fold.
>
> Compare the old graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f5a4
>
> to the new graph
>
> http://graphs.mikeal.couchone.com/#/graph/62b286fbb7aa55a4b0c4cc913c00f4d7
>
> I was under the impression that the Erlang IO subsystem was highly optimized but there seems to be no limit to perfection.
>
> NIFs are a giant black hole that will subsume Erlang code as performance has to be improved. Start at the lowest level and keep moving up. All that will be left of Erlang in the end is 99.99999% uptime, fault tolerance and supervision... of optimized C code. It's swell and I'm all for it!
>
> Patch is here:
>
> http://github.com/wagerlabs/couchdb/commit/23527eb8165f81e63d47b230f3297d3072c88d83
>
> ---
> http://twitter.com/wagerlabs
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>