[erlang-questions] fast file sending - erlang & nginx

[Steve Vinoski]

On Sun, Nov 21, 2010 at 7:15 AM, Rapsey <rapsey@REDACTED> wrote:
> I've done quite a bit of testing on this stuff and the end conclusion is
> that gen_tcp:send is simply slow. It uses way to much CPU than it should and
> I don't know why. I tried looking at the inet_drv.c stuff, but there is just
> to much code in there.

My experiences and measurements show pretty much the same thing. I
don't think gen_tcp necessarily targets top performance, but rather is
designed to be portable and tie cleanly into the whole inet framework,
but by doing so it suffers from unnecessary overhead in a variety of
places. Some of the problem areas we've observed are too many
{set,get}sockopt calls made that don't seem to add value, inet_db ets
lookups on each gen_tcp operation to find the socket driver, and some
handling of socket options/parameters in prim_inet that seems overly
expensive.

gen_tcp2, anyone?

Regarding the sendfile driver:

> On Sun, Nov 21, 2010 at 11:52 AM, Roberto Ostinelli <roberto@REDACTED>wrote:
>> instead, i tried using the sendfile system call developed by steve
>> vinosky sendfile_drv, packaged by tuncer

Vinoski, not Vinosky. :-)

>> [https://github.com/tuncer/sendfile], since in this way the file
>> reading is anyway done by a fast C system call and i would be able to
>> see if there are any differences. the results are that raw throughput
>> remains around the same.

That surprises me, but at the very least you should see much lower CPU
with the sendfile driver.

--steve