Hi all,<div><br></div><div>I first asked my question on <a href="http://stackoverflow.com/questions/10077269/erlang-accepting-ssl-connection-is-really-slow-comparing-to-c#comment12923311_10077269">stack overflow</a>. Unfortunately with no lock so far.</div>
<div>This is my last hope finding an clear answer. Here's it is :</div><div><br></div><div><div>I'm currently testing extreme condition on a piece of code written with Erlang.</div><div><br></div><div>I have implemented <a href="http://learnyousomeerlang.com/buckets-of-sockets#sockserv-revisited">learnyousomeerlang.com</a>'s technique of supervisor to have multiple accept capability.</div>
<div><br></div><div>Here the code slightly modified to handle SSL connections of the supervisor: </div><div><br></div><div> -module(mymodule).</div><div> </div><div> -behaviour(supervisor).</div><div> </div><div>
-export([start/0, start_socket/0]).</div><div> -define(SSL_OPTIONS, [{active, true},</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> {mode, list},</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> {reuseaddr, true},</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span> {cacertfile, "./ssl_key/server/gd_bundle.crt"},</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> {certfile, "./ssl_key/server/cert.pem"},</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span> {keyfile, "./ssl_key/server/key.pem"},</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> {password, "********"}</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span> ]).</div><div> </div><div> -export([init/1]).</div><div> </div><div> start_link() -></div><div> application:start(crypto),</div>
<div> crypto:start(),</div><div> application:start(public_key),</div><div> application:start(ssl),</div><div> supervisor:start_link({local, ?MODULE}, ?MODULE, []).</div><div> </div><div> init([]) -></div>
<div> {ok, LSocket} = ssl:listen(4242, ?SSL_OPTIONS),</div><div> spawn_link(fun empty_listeners/0),</div><div> {ok, {{simple_one_for_one, 60, 3600},</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> [{socket,</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span> {mymodule_serv, start_link, [LSocket]}, % pass the socket!</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span> temporary, 1000, worker, [mymodule_serv]}</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span> ]}}.</div><div> </div><div> empty_listeners() -></div><div> [start_socket() || _ <- lists:seq(1,100)],</div><div> ok.</div>
<div> </div><div> start_socket() -></div><div> supervisor:start_child(?MODULE, []).</div><div><br></div><div>Here's the code for gen_server which will represent every client connecting : </div><div><br>
</div><div> -module(mymodule_serv).</div><div> -behaviour(gen_server).</div><div> </div><div> -export([start_link/1]).</div><div> -export([init/1, handle_call/3, handle_cast/2, terminate/2, code_change/3, handle_info/2]).</div>
<div> </div><div> start_link(Socket) -></div><div> gen_server:start_link(?MODULE, Socket, []).</div><div> </div><div> init(Socket) -></div><div> gen_server:cast(self(), accept),</div><div> {ok, #client{socket=Socket, pid=self()}}.</div>
<div> </div><div> handle_call(_E, _From, Client) -></div><div> {noreply, Client}.</div><div> </div><div> handle_cast(accept, C = #client{socket=ListenSocket}) -></div><div> {ok, AcceptSocket} = ssl:transport_accept(ListenSocket),</div>
<div> mymodule:start_socket(),</div><div> ssl:ssl_accept(AcceptSocket),</div><div> ssl:setopts(AcceptSocket, [{active, true}, {mode, list}]),</div><div> {noreply, C#client{socket=AcceptSocket, state=connecting}}.</div>
<div> </div><div> [...]</div><div><br></div><div>I have the ability to launch close to 10.000 connections at once from multiple server.</div><div>While it will take 10 second to a ssl accepting bit of C++ code to accept all of them (which don't even have multiple accept pending), in Erlang this is quite different. It will accept at most 20 connections a second (according to netstat info, whilst C++ accept more like 1K connection per seconds)</div>
<div><br></div><div>While the 10K connections are awaiting for acceptance, I'm manually trying to connect as well.</div><div><br></div><div> openssl s_client -ssl3 -ign_eof -connect <a href="http://myserver.com:4242">myserver.com:4242</a></div>
<div><br></div><div>3 cases happen when I do :</div><div><br></div><div> - Connection simply timeout</div><div> - Connection will connect after waiting for it 30 sec. at least</div><div> - Connection will occur almost directly</div>
<div><br></div><div>When I try connecting manually with 2 consoles, the first done handshaking will not always be the first which tried to connect... Which I found particular.</div><div><br></div><div>The server configuration is : </div>
<div><br></div><div> - 2 x IntelŽ XeonŽ E5620 </div><div> - 8x 2.4GHz</div><div> - 24 Go RAM</div><div><br></div><div>I'm starting the Erlang shell with : </div><div><br></div><div> $erl +S 8:8</div><div><br></div>
<div>I have even tried to accept the connection with gen_tcp, and upgrading afterwards the connection to a SSL one. Still the same issue, it won't accept more than 10 connections a second... Is ssl:ssl_accept is doing this ? does it lock anything that would prevent Erlang to scale this ?</div>
<div><br></div><div>After looking around on other SSL server created in erlang, it seems that they use some kind of driver for SSL/TLS connection, my examples are RabbitMQ and EjabberD.</div><div>Nowhere there is ssl:ssl_accept in their Erlang code, I haven't investigate a lot, but it seems they have created their own driver in order to upgrade the TCP Socket to a SSL/TLS one.</div>
<div>Is that because there is an issue with Erlang's module SSL ? Does anyone know why they are using custom driver for SSL/TLS ?</div><div><br></div><div>Any thoughts on this ?</div></div><div><br></div><div>-- <br>Morgan<br>
<br>
</div>