[erlang-questions] Fwd: Unexpected behavior of HTTPC module - tls_connection process take more memory

Sanath Prasanna <>
Thu Oct 1 11:01:25 CEST 2015


Hi all,
Any update on this?
Br,
Robert

On Mon, Sep 28, 2015 at 5:12 PM, Sanath Prasanna <>
wrote:

> Hi all,
> Any update on this?
> Br,
> Robert
>
> On Fri, Sep 18, 2015 at 2:11 PM, Eranga Udesh <>
> wrote:
>
>> Hi,
>>
>> I am experiencing the same issue in,
>>
>>    - Erlang version : V6.4 , (Erlang/OTP 17 [erts-6.4] [source] [64-bit])
>>    - OS version/architecture (32/64 bit) : Red Hat Enterprise Linux
>>    Server release 6.6 (64 bit)
>>
>> The issue doesn't come always but after running the system for 3-10 days
>> can experience the tls_connection instances making high reductions and
>> consuming high memory as sent by Sanath. Since it's a production system, I
>> wasn't able to do much experiments, but applied the patches you sent, which
>> didn't solve this issue.
>>
>> There are 2 servers running Erlang VMs making SSL requests to the same
>> HTTPS host. When this issue happens, it happens in both the VMs. So it
>> could be triggered by some conditions in SSL or network connection. Even
>> though I left the VMs to run for a while to see if they can recover, but no
>> success. However if I restart the VMs, it starts to run as normal.
>>
>> So the summary is,
>>
>>    - It triggers by some condition in remote SSL host or network
>>    connection
>>    - Erlang VMs don't recover itself
>>    - Once restarted, it start working, which implies the issue is local.
>>    i.e. in Erlang VM (tls_connection)
>>
>> We will try to recreate it and send you more details. I wonder if anybody
>> else is experiencing such issue.
>>
>> Tks,
>> - Eranga
>>
>>
>>
>>
>>
>>
>> On Fri, Sep 18, 2015 at 1:36 PM, Ingela Andin <>
>> wrote:
>>
>>> Hi!
>>>
>>> As we are very busy with the release 18.1 I have not had time to try and
>>> recreate your problem. What version of OTP  and the ssl application are you
>>> using?
>>> Can you reproduce the problem with the latest on github?
>>>
>>> Regards Ingela Erlang/OTP Team - Ericsson AB
>>>
>>>
>>> On Thu, Sep 17, 2015 at 8:05 PM, Sanath Prasanna <>
>>> wrote:
>>>
>>>> Hi Ingela,
>>>> Any update on this??
>>>> "Even apply your patch, *still problem is persist.* Any
>>>> more suggestions to solve this unexpected behavior ? "
>>>> Br,
>>>> Robert
>>>>
>>>> On Thu, Sep 17, 2015 at 12:39 PM, Sanath Prasanna <
>>>> > wrote:
>>>>
>>>>> Hi Ingela,
>>>>> Even apply your patch, *still problem is persist.* Any
>>>>> more suggestions to solve this unexpected behavior ?
>>>>> Br,
>>>>> Robert
>>>>>
>>>>> On Tue, Sep 8, 2015 at 1:52 PM, Sanath Prasanna <>
>>>>> wrote:
>>>>>
>>>>>> Hi Ingela.
>>>>>> Tx a lot for your help & patch related to that.I'll inform you the
>>>>>> result after applying &  testing patch.
>>>>>> Br,
>>>>>> Robert
>>>>>>
>>>>>> On Tue, Sep 8, 2015 at 1:31 PM, Ingela Andin <>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> It could be an bug that in the ssl application that I just fixed.
>>>>>>> The default session cache
>>>>>>> was violating the API, and this in turn made the mechanism for not
>>>>>>> registering a lot of equivalent
>>>>>>> sessions in the client fail.
>>>>>>>
>>>>>>>  Here is the patch:
>>>>>>>
>>>>>>>
>>>>>>> diff --git a/lib/ssl/src/ssl_session.erl
>>>>>>> b/lib/ssl/src/ssl_session.erl
>>>>>>> index 1770faf..0d6cc93 100644
>>>>>>> --- a/lib/ssl/src/ssl_session.erl
>>>>>>> +++ b/lib/ssl/src/ssl_session.erl
>>>>>>> @@ -100,14 +100,14 @@ select_session([], _, _) ->
>>>>>>>      no_session;
>>>>>>>  select_session(Sessions, #ssl_options{ciphers = Ciphers}, OwnCert)
>>>>>>> ->
>>>>>>>      IsNotResumable =
>>>>>>> - fun([_Id, Session]) ->
>>>>>>> + fun(Session) ->
>>>>>>>   not (resumable(Session#session.is_resumable) andalso
>>>>>>>       lists:member(Session#session.cipher_suite, Ciphers)
>>>>>>>       andalso (OwnCert == Session#session.own_certificate))
>>>>>>>   end,
>>>>>>>      case lists:dropwhile(IsNotResumable, Sessions) of
>>>>>>>   [] ->   no_session;
>>>>>>> - [[Id, _]|_] -> Id
>>>>>>> + [Session | _] -> Session#session.session_id
>>>>>>>      end.
>>>>>>>
>>>>>>>  is_resumable(_, _, #ssl_options{reuse_sessions = false}, _, _, _,
>>>>>>> _) ->
>>>>>>> diff --git a/lib/ssl/src/ssl_session_cache.erl
>>>>>>> b/lib/ssl/src/ssl_session_cach
>>>>>>> e.erl
>>>>>>> index 11ed310..cfc48cd 100644
>>>>>>> --- a/lib/ssl/src/ssl_session_cache.erl
>>>>>>> +++ b/lib/ssl/src/ssl_session_cache.erl
>>>>>>> @@ -83,7 +83,7 @@ foldl(Fun, Acc0, Cache) ->
>>>>>>>
>>>>>>>  %%--------------------------------------------------------------------
>>>>>>>  select_session(Cache, PartialKey) ->
>>>>>>>      ets:select(Cache,
>>>>>>> -       [{{{PartialKey,'$1'}, '$2'},[],['$$']}]).
>>>>>>> +       [{{{PartialKey,'_'}, '$1'},[],['$1']}]).
>>>>>>>
>>>>>>>
>>>>>>>  %%--------------------------------------------------------------------
>>>>>>>  %%% Internal functions
>>>>>>>
>>>>>>>
>>>>>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>>>>>
>>>>>>>
>>>>>>> 2015-09-07 5:52 GMT+02:00 Sanath Prasanna <>:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I am running HTTP client using httpc Module to send both http and
>>>>>>>> https requests. normally sending arround 300 request per second without any
>>>>>>>> issue. however sometimes erlang node become very slow responsive. at that
>>>>>>>> time server load average is very high and using etop can identify
>>>>>>>> "tls_connection" process take more memory. when restart the erlang node its
>>>>>>>> become normal. as per my investigation normal time memory, processors,
>>>>>>>> loadAverage is not increasing. following is the HTTP request config
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> httpc:request(Method, Request, [{timeout, TimeoutTime}], [{sync,
>>>>>>>> false}])
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> below is the etop output at that time
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>         procs    1134                        processes 1504844
>>>>>>>> code         9309
>>>>>>>>
>>>>>>>>         runq        0                        atom          420
>>>>>>>> ets         29692
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Pid            Name or Initial Func    Time    Reds  Memory    MsgQ
>>>>>>>> Current Function
>>>>>>>>
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> <5490.26428.14>tls_connection:init/     '-'    733224580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26429.14>tls_connection:init/     '-'   1328524580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26430.14>tls_connection:init/     '-'    528924580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26431.14>tls_connection:init/     '-'   1432224580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26432.14>tls_connection:init/     '-'       024580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26433.14>tls_connection:init/     '-'       024580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26434.14>tls_connection:init/     '-'       024580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26435.14>tls_connection:init/     '-'       024580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>> <5490.26436.14>tls_connection:init/     '-'       024580768       0
>>>>>>>> gen_fsm:loop/7
>>>>>>>>
>>>>>>>>
>>>>>>>> can some one help me to solve this issue?
>>>>>>>>
>>>>>>>> Br,
>>>>>>>>
>>>>>>>> A.H.E. Robert
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> erlang-questions mailing list
>>>>>>>> 
>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> 
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> 
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151001/d3ed2b3b/attachment.html>


More information about the erlang-questions mailing list