fast file sending - erlang & nginx

[Roberto Ostinelli]

dear list,

i've just made a basic test to measure the sending of a small file
(5080 bytes) via http, using erlang and nginx. code for erlang's file
sending is appended here below. the results from ab benchmarks on an
ubuntu box show that the throughput of nginx is around 5 times the one
i can reach with erlang.

i've developed a fastcgi backend in erlang [soon to be released on
github], and using it i can reach comparable [if not faster]
throughtputs on it than the ones i can reach with nginx. so, the fact
that erlang is slower in sending static files cannot be due to the
http handling itself.

i decided to investigate if this was due to I/O reading speed.

i've seen that joel has done some research on this using NIF to
replace the file reading module on couchdb with the fd.erl module and
it's related NIFs
[https://github.com/wagerlabs/couchdb/tree/23527eb8165f81e63d47b230f3297d3072c88d83/src/couchdb],
however if i'm not wrong i've seen that he is not using this module
anymore, so i decided not test it.

instead, i tried using the sendfile system call developed by steve
vinosky sendfile_drv, packaged by tuncer
[https://github.com/tuncer/sendfile], since in this way the file
reading is anyway done by a fast C system call and i would be able to
see if there are any differences. the results are that raw throughput
remains around the same.

therefore, i believe that the difference in throughput between nginx
and this erlang code is not due to I/O issues. i'm wandering if nginx
is nmapping files into memory, which would allow it to outperform in
this way, or if i am missing a point here.

i'd be glad to have pointers on how to improve fast file sending over http.

cheers,

r.

=========================================
-module(send_file).
-include_lib("kernel/include/file.hrl").
-compile(export_all).

% sending of a file
file_send(Socket, FilePath, Headers) ->
	% get file size
	case file:read_file_info(FilePath) of
		{ok, FileInfo} ->
			% get filesize
			FileSize = FileInfo#file_info.size,
			% do the gradual sending
			file_open_and_send(Socket, FilePath);
		{error, Reason} ->
			{error, Reason}
	end.
file_open_and_send(Socket, FilePath) ->
	case file:open(FilePath, [read, binary]) of
		{error, Reason} ->
			{error, Reason};
		{ok, IoDevice} ->
			% read portions
			case file_read_and_send(Socket, IoDevice, 0) of
				{error, Reason} ->
					file:close(IoDevice),
					{error, Reason};
				ok ->
					file:close(IoDevice),
					ok
			end
	end.
file_read_and_send(Socket, IoDevice, Position) ->
	% read buffer
	case file:pread(IoDevice, Position, ?FILE_READ_BUFFER) of
		{ok, Data} ->
			% file read, send
			gen_tcp:send(Socket, Data),
			% loop
			file_read_and_send(Socket, IoDevice, Position + ?FILE_READ_BUFFER);
		eof ->
			% finished
			ok;
		{error, Reason} ->
			{error, Reason}
	end.