<div dir="ltr">Hi,<div><br></div><div>I used to face this kind of problem when I updated erlang from OTP 17.x.x to 18 but I think the problem was fixed already in 18.2</div><div><br></div><div>Here is what I found out:</div><div><br></div><div>My problem was that `ets` table keeps growing and it is caused by `server_ssl_otp_session_cache`</div>
<p class=""><span class="">I traced it back to the source code of Erlang 17 & Erlang 18 release branch, in ssl module/ssl_manager.erl. There was a mistake in calculation of 24H_in_msec in Erlang 17.</span></p><p class=""><span class=""><br></span></p><p class=""><span class="">`Erlang OTP 17`</span></p>
<p class=""><span class=""><br></span></p><p class=""><span class="">-define('24H_in_msec', <font color="#ff0000">8640000</font>).</span></p>
<p class=""><span class="">-define('24H_in_sec', 8640).</span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p>
<p class=""><span class=""></span><br></p>
<p class=""><span class="">`Erlang OTP 18`</span></p>
<p class=""><span class=""></span><br></p>
<p class=""><span class="">-define('24H_in_msec', <font color="#ff0000">86400000</font>).</span></p>
<p class=""><span class="">-define('24H_in_sec', 86400).</span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p><p class=""><span class=""><br></span></p><p class=""><span class="">So you can try to pass </span> `session_lifetime` to 30mn (1800 seconds) to `ssl` application (module).</p><p class="">Sethy</p>







</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 28, 2016 at 5:36 PM,  <span dir="ltr"><<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
Could you please clarify one moment. Is the ssl session cache used only<br>
when reuse_sessions is set to true? Is there a way to turn off the cache<br>
completely? I'm trying to figure out what settings are needed for<br>
thousands of short-lived connections.<br>
<br>
28/04/16 12:21, Ingela Andin пишет:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi!<br>
<br>
<br>
2016-04-27 15:44 GMT+02:00 <<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a> <mailto:<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a>>>:<span class=""><br>
<br>
    Hello,<br>
<br>
    The reason was session_cache_server_max (defaults to 1000).<br>
    The ssl_manager quickly get filled by<br>
<br>
    {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}}<br>
    {delayed_clean_session,{8443,<<14,50,229,...>>},40986}<br>
<br>
    messages.<br>
<br>
I see, we will looking into mitigating this effect.<br>
<br>
Regards Ingela Erlang/OTP team - Ericsson AB<br>
<br>
<br>
<br>
    27/04/16 13:29, Ingela Andin пишет:<br>
<br>
        Hi!<br>
<br>
        You could try using the  observer application to find out more<br>
        information about your system state.<br>
        There is no apparent reason why ssl-7.3 should consume a lot<br>
        more memory<br>
        than ssl-6.0.1.2.<br>
        But a lot has append between the versions and heavy load may<br>
        find corner<br>
        cases that where missed.<br>
<br>
        Regards Ingela Erlang/OTP Team - Ericsson AB<br>
<br>
<br>
        2016-04-25 14:35 GMT+02:00 <<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a><br></span>
        <mailto:<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a>> <mailto:<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a><div><div class="h5"><br>
        <mailto:<a href="mailto:vicbaz@yandex.ru" target="_blank">vicbaz@yandex.ru</a>>>>:<br>
<br>
<br>
             Hello,<br>
<br>
             When I run<br>
        <a href="https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world" rel="noreferrer" target="_blank">https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world</a><br>
             under load (using <a href="https://github.com/JoeDog/siege" rel="noreferrer" target="_blank">https://github.com/JoeDog/siege</a>) I found<br>
        a big<br>
             difference in memory consumption between OTP-17.5 and<br>
        OTP-18.3. I know<br>
             the ssl application has many changes in new versions. Is<br>
        this memory<br>
             usage normal now or can be reduced using some ssl<br>
        application settings?<br>
<br>
             I use OTP-17.5.6.9 and OTP-18.3.1.<br>
<br>
<br>
        ================================================================================<br>
             Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2]<br>
             [async-threads:10] [hipe] [kernel-poll:false]<br>
<br>
             1> application:which_applications().<br>
             [{ssl_hello_world,"Cowboy Hello World example with SSL.",<br>
                                "1"},<br>
               {cowboy,"Small, fast, modular HTTP server.","1.0.4"},<br>
               {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"},<br>
               {cowlib,"Support library for manipulating Web protocols.",<br>
                       "1.0.2"},<br>
               {ssl,"Erlang/OTP SSL application","6.0.1.2"},<br>
               {public_key,"Public key infrastructure","0.23"},<br>
               {crypto,"CRYPTO","3.5"},<br>
               {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"},<br>
               {stdlib,"ERTS  CXC 138 10","2.4"},<br>
               {kernel,"ERTS  CXC 138 10","3.2.0.1"}]<br>
<br>
             2> erlang:memory().<br>
             [{total,43913840},<br>
               {processes,8036312},<br>
               {processes_used,8030744},<br>
               {system,35877528},<br>
               {atom,470537},<br>
               {atom_used,455904},<br>
               {binary,1498800},<br>
               {code,11074476},<br>
               {ets,14516424}]<br>
<br>
             3> recon_alloc:memory(used).<br>
             45519464<br>
<br>
             4> recon_alloc:memory(usage).<br>
             0.7212344918526264<br>
<br>
             5> recon:proc_window(memory, 10, 500).<br>
             [{<0.25910.2>,68120,<br>
                [{current_function,{gen_fsm,loop,7}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.25908.2>,68120,<br>
                [{current_function,{gen_fsm,loop,7}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.284.0>,15808,<br>
                [ssl_manager,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.285.0>,12712,<br>
                [tls_connection_sup,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.25909.2>,5888,<br>
                [{current_function,{gen,do_call,4}},<br>
                 {initial_call,{cowboy_protocol,init,4}}]},<br>
               {<0.324.0>,4888,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.302.0>,4808,<br>
                [{current_function,{ranch_conns_sup,loop,4}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.330.0>,3016,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.25911.2>,2848,<br>
                [{current_function,{gen,do_call,4}},<br>
                 {initial_call,{cowboy_protocol,init,4}}]},<br>
               {<0.340.0>,1872,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]}]<br>
<br>
             6> recon:proc_window(reductions, 10, 500).<br>
             [{<0.2634.3>,11161,<br>
                [{current_function,{gen_fsm,loop,7}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.2633.3>,9747,<br>
                [{current_function,{crypto,int_to_bin_pos,2}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.284.0>,4097,<br>
                [ssl_manager,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.285.0>,3314,<br>
                [tls_connection_sup,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.302.0>,1003,<br>
                [{current_function,{ranch_conns_sup,loop,4}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.378.0>,426,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.372.0>,425,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.379.0>,425,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.377.0>,425,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.343.0>,425,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]}]<br>
<br>
             RES in htop ~50-60M<br>
<br>
<br>
        ================================================================================<br>
<br>
             Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2]<br>
             [async-threads:10] [hipe] [kernel-poll:false]<br>
<br>
             1> application:which_applications().<br>
             [{ssl_hello_world,"Cowboy Hello World example with SSL.",<br>
                                "1"},<br>
               {cowboy,"Small, fast, modular HTTP server.","1.0.4"},<br>
               {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"},<br>
               {cowlib,"Support library for manipulating Web protocols.",<br>
                       "1.0.2"},<br>
               {ssl,"Erlang/OTP SSL application","7.3"},<br>
               {public_key,"Public key infrastructure","1.1.1"},<br>
               {crypto,"CRYPTO","3.6.3"},<br>
               {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"},<br>
               {recon,"Diagnostic tools for production use","2.3.1"},<br>
               {stdlib,"ERTS  CXC 138 10","2.8"},<br>
               {kernel,"ERTS  CXC 138 10","4.2"}]<br>
<br>
             2> erlang:memory().<br>
             [{total,949153560},<br>
               {processes,902969888},<br>
               {processes_used,902558312},<br>
               {system,46183672},<br>
               {atom,437761},<br>
               {atom_used,432875},<br>
               {binary,48320},<br>
               {code,11407283},<br>
               {ets,883640}]<br>
<br>
             3> recon_alloc:memory(used).<br>
             863194560<br>
<br>
             4> recon_alloc:memory(usage).<br>
             0.8005009562139471<br>
<br>
             5> recon:proc_window(memory, 10, 500).<br>
             [{<0.290.0>,94360,<br>
                [ssl_manager,<br>
                 {current_function,{ssl_manager,handle_cast,2}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.13589.3>,24640,<br>
                [{current_function,{ssl_manager,validate_session,3}},<br>
                 {initial_call,{ssl_manager,init_session_validator,1}}]},<br>
               {<0.291.0>,12832,<br>
                [tls_connection_sup,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.13590.3>,10960,<br>
                [{current_function,{gen,do_call,4}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.318.0>,7904,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.308.0>,3056,<br>
                [{current_function,{ranch_conns_sup,loop,4}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.385.0>,1872,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.254.0>,0,<br>
                [code_server,<br>
                 {current_function,{code_server,loop,1}},<br>
                 {initial_call,{erlang,apply,2}}]},<br>
               {<0.325.0>,0,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.357.0>,0,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]}]<br>
<br>
             6> recon:proc_window(reductions, 10, 500).<br>
             [{<0.290.0>,232358,<br>
                [ssl_manager,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.291.0>,3058,<br>
                [tls_connection_sup,<br>
                 {current_function,{gen_server,loop,6}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.308.0>,1129,<br>
                [{current_function,{ranch_conns_sup,loop,4}},<br>
                 {initial_call,{proc_lib,init_p,5}}]},<br>
               {<0.339.0>,435,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.340.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.355.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.353.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.334.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.366.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]},<br>
               {<0.360.0>,430,<br>
                [{current_function,{prim_inet,accept0,2}},<br>
                 {initial_call,{ranch_acceptor,loop,3}}]}]<br>
<br>
             RES in htop ~900-1000M<br>
             _______________________________________________<br>
             erlang-questions mailing list<br>
        <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a> <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>><br></div></div>
        <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
        <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>>><br>
        <a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
<br>
</blockquote><span class="">
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
</span><a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote></div><br></div>