[erlang-questions] Process/FD leak in SSL R15B01

Loïc Hoguin essen@REDACTED
Mon Oct 15 17:17:54 CEST 2012


Hello,

We are observing a constant increase in the number of ssl_connection 
processes. We have investigated the problem and are not sure how to go 
about it further, considering the results.

We switched from using ssltunnel to directly speaking to the Erlang 
nodes in SSL for speaking with our mobile applications.

We observed that we have right now around 25000 sockets in ESTABLISHED 
(increasing over time) and around the same in FIN_WAIT2 (also increasing 
over time). In total it increases at about 10000 sockets per day.

Looking at the applications running it appears only the SSL application 
has extra processes compared to normal. The other parts of the system 
appear to be functioning normally.

What's more alarming is that there is a lot more SSL connection 
processes than actual connection processes in Cowboy (between 10-15K and 
not increasing over time).

Investigating the SSL application to find the source, I ended up 
applying the following snippet to the list of PIDs supervised by 
ssl_connection_sup:

 > lists:foldl(fun(X, Sum) -> case erlang:process_info(X) of undefined 
-> Sum; [{current_function, XXX}|_] -> case lists:keyfind(XXX, 1, Sum) 
of false -> Curr = 0; {_, Curr} -> ok end, lists:keystore(XXX, 1, Sum, 
{XXX, Curr + 1}) end end, [], List).
[{{prim_inet,recv0,3},25856},{{gen_fsm,loop,7},26574}]

Not sure which one is the ESTABLISHED list and which one is the 
FIN_WAIT2. Of course, I can't use sys:get_status/1 on the PIDs stuck in 
prim_inet:recv0/3 because the receive there is quite specific. So I 
can't get the stacktrace. The other case doesn't seem to give anything 
useful (for my level of knowledge, anyway).

The process that owns the SSL socket died. So the ssl_connection process 
must have received something. Problem is, it didn't close the socket 
properly, and the ssl_connection process still runs when it shouldn't.

Is it a known issue? Is there a patch available? If not, how should I go 
about fixing it?

Thanks.

-- 
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu



More information about the erlang-questions mailing list