Sendfile in erlang

Per Bergqvist per@REDACTED
Mon Nov 17 07:14:15 CET 2003


Just a note if you consider introducing this as standard in OTP...
(just hoping ...)

I did a similiar thing a couple of years ago with similiar results.

The difference was that I introduces a new send directive to 
gen_tcp which allowed to mixing gen_tcp:send and gen_tcp:sendfile call
on the socket.

Also added a send_urgent directive to send urgent tcp data ...


> Hi,
> I'm not sure whether the proper list is e-questions or e-patches,
but this
> is a small patch anyway.
> This draft patch adds a sendfile() interface to Erlang. Sendfile(2)
is a
> system call present in Linux 2.2, AIX 5 and later kernels, and
> interfaces are present in recent versions of Solaris and possibly
> Unices. The usual loop of read()ing from a file and then send()ing
to a
> socket or write()ing to another file has an unnecesarily large
> copying data from kernel space to user space on read, and then back
> kernel space again on write() or send(). Besides, if we are reading
> Erlang, that means getting all those data chunks into the erlang
> memory management system only to get them out again immediately and
> GC them sometime in the future. Very often (think of a web or file
> our program has no use for that read data except sending it out
> Sendfile(f,t,o,c) simply instructs the kernel (the OS kernel, not
> $ROOTDIR/lib/kernel) to read c bytes at offset o of file descriptor
f and
> write them again to file descriptor t. No data is moved to/from user
> space.
> ObPerfData: a cycle of file:read() and gen_tcp:send() moving 4KB
> over 1000Base-T between 1GHz Pentium3 machines sustains a throughput
> about 55Mbps. A cycle of file:sendfile() calls sustains over 410Mbps
> the pipe. Make sure you have a well supported network card before
> The patch is for testing purposes - I'd be glad to hear comments. I
> kept the kernel sendfile semantics: it may write less bytes than
> requested, just like the send(2) syscall; return value is {ok,
> or {error, Reason}. Maybe it would be more polite to behave like
> gen_tcp:send instead and make sure all data is sent, or else return
> error. More ugly details: it needs the socket descriptor *number*,
so for
> now you have to call the undocumented function get_fd in prim_inet.
> example:
>     {ok,From}=file:open(Filename,[read,raw]),
>     {ok,Sock}=gen_tcp:connect(Host,Port,[binary,{packet,0}]),
>     {ok,SockFD}=prim_inet:getfd(Sock),
>     {ok,Sent}=file:sendfile(From,SockFD,Pos,Block),
> No guarantees, backup first, parachute not included, etc.
> Regards,
> Miguel

More information about the erlang-questions mailing list