<div dir="ltr"><div>Hi,<br><br></div>I am experiencing the same issue in,<br><ul><li>Erlang version : V6.4 , (Erlang/OTP 17 [erts-6.4] [source] [64-bit])</li><li>OS version/architecture (32/64 bit) : Red Hat Enterprise Linux Server release 6.6 (64 bit)</li></ul><p>The issue doesn't come always but after running the system for 3-10 days can experience the tls_connection instances making high reductions and consuming high memory as sent by Sanath. Since it's a production system, I wasn't able to do much experiments, but applied the patches you sent, which didn't solve this issue.</p><p>There are 2 servers running Erlang VMs making SSL requests to the same HTTPS host. When this issue happens, it happens in both the VMs. So it could be triggered by some conditions in SSL or network connection. Even though I left the VMs to run for a while to see if they can recover, but no success. However if I restart the VMs, it starts to run as normal.<br></p><p>So the summary is,</p><ul><li>It triggers by some condition in remote SSL host or network connection</li><li>Erlang VMs don't recover itself</li><li>Once restarted, it start working, which implies the issue is local. i.e. in Erlang VM (tls_connection)</li></ul><p>We will try to recreate it and send you more details. I wonder if anybody else is experiencing such issue.</p><p>Tks,<br>- Eranga<br></p><p><br></p><p><br></p><p><br></p><p><br></p></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Sep 18, 2015 at 1:36 PM, Ingela Andin <span dir="ltr"><<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">Hi!<div><br></div><div>As we are very busy with the release 18.1 I have not had time to try and recreate your problem. What version of OTP and the ssl application are you using?</div><div>Can you reproduce the problem with the latest on github?</div><div><br></div><div>Regards Ingela Erlang/OTP Team - Ericsson AB</div><div><div class="h5"><div><div><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 17, 2015 at 8:05 PM, Sanath Prasanna <span dir="ltr"><<a href="mailto:ahe.sanath@gmail.com" target="_blank">ahe.sanath@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="font-size:12.8px">Hi Ingela,</span><div>Any update on this?? <span><br style="font-size:12.8px"><div style="font-size:12.8px"><span style="font-size:12.8px">"Even apply your patch, <b>still problem is persist.</b> Any more suggestions to solve this unexpected behavior ? "</span></div><div style="font-size:12.8px"><span style="font-size:12.8px">Br,</span></div><div style="font-size:12.8px"><span style="font-size:12.8px">Robert</span></div></span></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 17, 2015 at 12:39 PM, Sanath Prasanna <span dir="ltr"><<a href="mailto:ahe.sanath@gmail.com" target="_blank">ahe.sanath@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="font-size:12.8px">Hi Ingela,</span><br><div><span style="font-size:12.8px">Even apply your patch, <b>still problem is persist.</b> Any more suggestions to solve this unexpected behavior ? </span></div><div><span style="font-size:12.8px">Br,</span></div><div><span style="font-size:12.8px">Robert</span></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 8, 2015 at 1:52 PM, Sanath Prasanna <span dir="ltr"><<a href="mailto:ahe.sanath@gmail.com" target="_blank">ahe.sanath@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Ingela.<div>Tx a lot for your help & patch related to that.I'll inform you the result after applying & testing patch.</div><div>Br,</div><div>Robert</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 8, 2015 at 1:31 PM, Ingela Andin <span dir="ltr"><<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi!<div><br></div><div>It could be an bug that in the ssl application that I just fixed. The default session cache</div><div>was violating the API, and this in turn made the mechanism for not registering a lot of equivalent</div><div>sessions in the client fail.</div><div><br></div><div> Here is the patch:</div><div><br></div><div><br></div><div><div>diff --git a/lib/ssl/src/ssl_session.erl b/lib/ssl/src/ssl_session.erl</div><div>index 1770faf..0d6cc93 100644</div><div>--- a/lib/ssl/src/ssl_session.erl</div><div>+++ b/lib/ssl/src/ssl_session.erl</div><div>@@ -100,14 +100,14 @@ select_session([], _, _) -></div><div> no_session;</div><div> select_session(Sessions, #ssl_options{ciphers = Ciphers}, OwnCert) -></div><div> IsNotResumable =</div><div>-<span style="white-space:pre-wrap"> </span>fun([_Id, Session]) -></div><div>+<span style="white-space:pre-wrap"> </span>fun(Session) -></div><div> <span style="white-space:pre-wrap"> </span>not (resumable(Session#session.is_resumable) andalso</div><div> <span style="white-space:pre-wrap"> </span> lists:member(Session#session.cipher_suite, Ciphers)</div><div> <span style="white-space:pre-wrap"> </span> andalso (OwnCert == Session#session.own_certificate))</div><div> <span style="white-space:pre-wrap"> </span>end,</div><div> case lists:dropwhile(IsNotResumable, Sessions) of</div><div> <span style="white-space:pre-wrap"> </span>[] -> no_session;</div><div>-<span style="white-space:pre-wrap"> </span>[[Id, _]|_] -> Id</div><div>+<span style="white-space:pre-wrap"> </span>[Session | _] -> Session#session.session_id</div><div> end.</div><div> </div><div> is_resumable(_, _, #ssl_options{reuse_sessions = false}, _, _, _, _) -></div><div>diff --git a/lib/ssl/src/ssl_session_cache.erl b/lib/ssl/src/ssl_session_cach</div><div>e.erl</div><div>index 11ed310..cfc48cd 100644</div><div>--- a/lib/ssl/src/ssl_session_cache.erl</div><div>+++ b/lib/ssl/src/ssl_session_cache.erl</div><div>@@ -83,7 +83,7 @@ foldl(Fun, Acc0, Cache) -></div><div> %%--------------------------------------------------------------------</div><div> select_session(Cache, PartialKey) -> </div><div> ets:select(Cache, </div><div>-<span style="white-space:pre-wrap"> </span> [{{{PartialKey,'$1'}, '$2'},[],['$$']}]).</div><div>+<span style="white-space:pre-wrap"> </span> [{{{PartialKey,'_'}, '$1'},[],['$1']}]).</div><div> </div><div> %%--------------------------------------------------------------------</div><div> %%% Internal functions</div></div><div><br></div><div><br></div><div>Regards Ingela Erlang/OTP team - Ericsson AB</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>2015-09-07 5:52 GMT+02:00 Sanath Prasanna <span dir="ltr"><<a href="mailto:ahe.sanath@gmail.com" target="_blank">ahe.sanath@gmail.com</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><p class="MsoNormal">Hi all,</p><p class="MsoNormal">I am running HTTP client using httpc Module to send both
http and https requests. normally sending arround 300 request per second
without any issue. however sometimes erlang node become very slow responsive.
at that time server load average is very high and using etop can identify
"tls_connection" process take more memory. when restart the erlang
node its become normal. as per my investigation normal time memory, processors,
loadAverage is not increasing. following is the HTTP request config</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">httpc:request(Method, Request, [{timeout, TimeoutTime}],
[{sync, false}])</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">below is the etop output at that time</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">
procs
1134
processes 1504844 code
9309</p>
<p class="MsoNormal">
runq
0
atom
420 ets 29692</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Pid
Name or Initial Func Time Reds
Memory MsgQ Current Function</p>
<p class="MsoNormal">----------------------------------------------------------------------------------------</p>
<p class="MsoNormal"><5490.26428.14>tls_connection:init/
'-' 733224580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26429.14>tls_connection:init/
'-' 1328524580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26430.14>tls_connection:init/
'-' 528924580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26431.14>tls_connection:init/
'-' 1432224580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26432.14>tls_connection:init/
'-'
024580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26433.14>tls_connection:init/
'-'
024580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26434.14>tls_connection:init/
'-' 024580768
0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26435.14>tls_connection:init/
'-'
024580768 0
gen_fsm:loop/7 </p>
<p class="MsoNormal"><5490.26436.14>tls_connection:init/
'-'
024580768 0
gen_fsm:loop/7 </p><p class="MsoNormal"><br></p><p class="MsoNormal">can some one help me to solve this issue? </p><p class="MsoNormal">Br,</p><p class="MsoNormal">A.H.E. Robert </p></div>
<br></div></div><span>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div></div></div></div>
</div><br></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>