Sendfile in erlang

[Per Bergqvist]

Hi,

Just a note if you consider introducing this as standard in OTP...
(just hoping ...)

I did a similiar thing a couple of years ago with similiar results.

The difference was that I introduces a new send directive to 
gen_tcp which allowed to mixing gen_tcp:send and gen_tcp:sendfile call
on the socket.

Also added a send_urgent directive to send urgent tcp data ...

/Per

-------------------
> 
> Hi,
> 
> I'm not sure whether the proper list is e-questions or e-patches,
but this
> is a small patch anyway.
> 
> This draft patch adds a sendfile() interface to Erlang. Sendfile(2)
is a
> system call present in Linux 2.2, AIX 5 and later kernels, and
similar
> interfaces are present in recent versions of Solaris and possibly
other
> Unices. The usual loop of read()ing from a file and then send()ing
to a
> socket or write()ing to another file has an unnecesarily large
overhead:
> copying data from kernel space to user space on read, and then back
to
> kernel space again on write() or send(). Besides, if we are reading
from
> Erlang, that means getting all those data chunks into the erlang
runtime
> memory management system only to get them out again immediately and
then
> GC them sometime in the future. Very often (think of a web or file
server)
> our program has no use for that read data except sending it out
again.
> 
> Sendfile(f,t,o,c) simply instructs the kernel (the OS kernel, not
> $ROOTDIR/lib/kernel) to read c bytes at offset o of file descriptor
f and
> write them again to file descriptor t. No data is moved to/from user
> space.
> 
> 
> ObPerfData: a cycle of file:read() and gen_tcp:send() moving 4KB
chunks
> over 1000Base-T between 1GHz Pentium3 machines sustains a throughput
of
> about 55Mbps. A cycle of file:sendfile() calls sustains over 410Mbps
down
> the pipe. Make sure you have a well supported network card before
trying.
> 
> The patch is for testing purposes - I'd be glad to hear comments. I
have
> kept the kernel sendfile semantics: it may write less bytes than
> requested, just like the send(2) syscall; return value is {ok,
SentBytes}
> or {error, Reason}. Maybe it would be more polite to behave like
> gen_tcp:send instead and make sure all data is sent, or else return
an
> error. More ugly details: it needs the socket descriptor *number*,
so for
> now you have to call the undocumented function get_fd in prim_inet.
An
> example:
> 
>     {ok,From}=file:open(Filename,[read,raw]),
>     {ok,Sock}=gen_tcp:connect(Host,Port,[binary,{packet,0}]),
>     {ok,SockFD}=prim_inet:getfd(Sock),
>     {ok,Sent}=file:sendfile(From,SockFD,Pos,Block),
> 
> 
> No guarantees, backup first, parachute not included, etc.
> 
> Regards,
> 
> Miguel
>