Inets performance

[Ulf Wiger]

On Tue, 6 Mar 2001, Miguel Barreiro Paz wrote:

>Under high throughput demands, Inets
>will suffer badly from its higher cpu overhead compared to Apache (who
>quickly gets I/O bound before processor is a bottleneck, unless you abuse
>quite a bit of rewriting rules and such).

Inets is in dire need of optimization.



>I suppose Apache on Linux nowadays will serve html files with
>sendfile(2) directly if it can (ie., unless it has to parse shtml or
>so), thus avoiding a copy, a userspace buffer, and at least two
>kernel-userspace switches per write. That is close to as fast as you
>can get without hardcore kernelHTTPd solutions :)

Inets could at least use a linked-in driver that chunked a file and
delivered the chunks directly to a socket.

The file.erl module is really somewhat optimized for fairness (to
erlang processes not using heavy I/O) and pays extra reductions for
each message coming from the driver. If you want to write some
application that has really fast file I/O, you can't use file.erl as
it works today (as mnesia has found out...)

>I my very limited experience with inets, I have found a bit limiting
>that it can't serve streams, only finite-length data. And as far as
>I remember it won't serve anything produced by an erlet until the
>erlet has finished its output.

When I wrote my source code viewer (the ccviewer contrib), I
implemented such a function. I didn't use Inets, but I'm sure the same
thing could be done there. Here's the code: ccv_html_lib:progress(Fmt,
Args) mimics io:format/2 and <herecy>uses the process dictionary to
keep track of the socket, and to cache the complete page</herecy>.
When done, the request thread calls maybe_save_progress(). The reason
for this was that I use this for operations that can take so long that
the Netscape client often times out. In such a case (or if the user
closes the window), the complete page is stored an can be fetched
later.

/Uffe

progress(Fmt, Args) ->
    progress(Fmt, Args, get_socket()).


progress(Fmt, Args, Socket) ->
    Str = io_lib:format(Fmt, Args),
    to_socket(Socket, Str),
    store_progress(Str),
    io:format("~p:" ++ Fmt, [self()|Args]).

to_socket(no_socket, _) -> ok;
to_socket(Socket, Str) ->
    case send(Socket, Str) of 
        {error, Reason} ->
            put(progress_to_socket, closed);
        _ ->
            ok
    end.

store_progress(Str) ->
    Sofar = get_html_progress(),
    put(html_progress, [Str | Sofar]).

get_html_progress() ->
    case get(html_progress) of
        undefined ->
            [];
        Other ->
            Other
    end.

maybe_save_progress() ->
    Title = get_strict(progress_title),
    User = get_strict(progress_user),
    Key = get_strict(progress_key),
    case get_socket() of
        no_socket ->
            case get(html_progress) of
                undefined ->
                    ok;
                HtmlRev ->
                    Bin = list_to_binary(["<pre>\n", 
                                          lists:reverse(HtmlRev),
                                         "\n</pre>"]),
                    ccv_user:store_page(Key, Title, User, Bin)
            end;
        Socket ->
            ok
    end.

get_strict(Key) ->
    case get(Key) of
        undefined ->
            exit(unexpected);
        Other ->
            Other
    end.



-- 
Ulf Wiger                                    tfn: +46  8 719 81 95
Senior System Architect                      mob: +46 70 519 81 95
Strategic Product & System Management    ATM Multiservice Networks
Data Backbone & Optical Services Division      Ericsson Telecom AB