[erlang-questions] Erlang accepting SSL connection is really slow (comparing to C++)

[SEGALIS Morgan]

Hi all,

I first asked my question on stack
overflow<http://stackoverflow.com/questions/10077269/erlang-accepting-ssl-connection-is-really-slow-comparing-to-c#comment12923311_10077269>.
Unfortunately with no lock so far.
This is my last hope finding an clear answer. Here's it is :

I'm currently testing extreme condition on a piece of code written with
Erlang.

I have implemented
learnyousomeerlang.com<http://learnyousomeerlang.com/buckets-of-sockets#sockserv-revisited>'s
technique of supervisor to have multiple accept capability.

Here the code slightly modified to handle SSL connections of the
supervisor:

    -module(mymodule).

    -behaviour(supervisor).

    -export([start/0, start_socket/0]).
    -define(SSL_OPTIONS, [{active, true},
          {mode, list},
          {reuseaddr, true},
          {cacertfile, "./ssl_key/server/gd_bundle.crt"},
          {certfile, "./ssl_key/server/cert.pem"},
          {keyfile, "./ssl_key/server/key.pem"},
          {password, "********"}
         ]).

    -export([init/1]).

    start_link() ->
        application:start(crypto),
        crypto:start(),
        application:start(public_key),
        application:start(ssl),
        supervisor:start_link({local, ?MODULE}, ?MODULE, []).

    init([]) ->
        {ok, LSocket} = ssl:listen(4242, ?SSL_OPTIONS),
        spawn_link(fun empty_listeners/0),
        {ok, {{simple_one_for_one, 60, 3600},
      [{socket,
        {mymodule_serv, start_link, [LSocket]}, % pass the socket!
        temporary, 1000, worker, [mymodule_serv]}
      ]}}.

    empty_listeners() ->
        [start_socket() || _ <- lists:seq(1,100)],
        ok.

    start_socket() ->
        supervisor:start_child(?MODULE, []).

Here's the code for gen_server which will represent every client connecting
:

    -module(mymodule_serv).
    -behaviour(gen_server).

    -export([start_link/1]).
    -export([init/1, handle_call/3, handle_cast/2, terminate/2,
code_change/3, handle_info/2]).

    start_link(Socket) ->
        gen_server:start_link(?MODULE, Socket, []).

    init(Socket) ->
        gen_server:cast(self(), accept),
        {ok, #client{socket=Socket, pid=self()}}.

    handle_call(_E, _From, Client) ->
        {noreply, Client}.

    handle_cast(accept, C = #client{socket=ListenSocket}) ->
        {ok, AcceptSocket} = ssl:transport_accept(ListenSocket),
        mymodule:start_socket(),
        ssl:ssl_accept(AcceptSocket),
        ssl:setopts(AcceptSocket, [{active, true}, {mode, list}]),
        {noreply, C#client{socket=AcceptSocket, state=connecting}}.

    [...]

I have the ability to launch close to 10.000 connections at once from
multiple server.
While it will take 10 second to a ssl accepting bit of C++ code to accept
all of them (which don't even have multiple accept pending), in Erlang this
is quite different. It will accept at most 20 connections a second
(according to netstat info, whilst C++ accept more like 1K connection per
seconds)

While the 10K connections are awaiting for acceptance, I'm manually trying
to connect as well.

    openssl s_client -ssl3 -ign_eof -connect myserver.com:4242

3 cases happen when I do :

 - Connection simply timeout
 - Connection will connect after waiting for it 30 sec. at least
 - Connection will occur almost directly

When I try connecting manually with 2 consoles, the first done handshaking
will not always be the first which tried to connect... Which I found
particular.

The server configuration is :

 - 2 x Intel® Xeon® E5620
 - 8x 2.4GHz
 - 24 Go RAM

I'm starting the Erlang shell with :

    $erl +S 8:8

I have even tried to accept the connection with gen_tcp, and upgrading
afterwards the connection to a SSL one. Still the same issue, it won't
accept more than 10 connections a second... Is ssl:ssl_accept is doing this
? does it lock anything that would prevent Erlang to  scale this ?

After looking around on other SSL server created in erlang, it seems that
they use some kind of driver for SSL/TLS connection, my examples are
RabbitMQ and EjabberD.
Nowhere there is ssl:ssl_accept in their Erlang code, I haven't investigate
a lot, but it seems they have created their own driver in order to upgrade
the TCP Socket to a SSL/TLS one.
Is that because there is an issue with Erlang's module SSL ? Does anyone
know why they are using custom driver for SSL/TLS ?

Any thoughts on this ?

-- 
Morgan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120410/8922051d/attachment.htm>