<div dir="ltr">Dear all,<div><br></div><div>Finally I know what is the difference between Erlang 17 and Erlang 18 in ssl module which caused a problem of memory. It is not really a memory leak but simply the caching problem.</div><div><br></div><div>I take a look at the source code of ssl module (ssl_manager.erl) in OTP-17 & OTP-18. It is how they define the 24h in seconds. In OTP-17, somebody did a wrong multiplication. While in OTP-18, the calculations were correct for both 24h-in-sec & 24h-in-msec!</div><div><br></div><div>By the way, thanks to Ingela, Loïc and Jesper for helping me finding the problems and solution.</div><div><br></div><div>Thanks,</div><div>T.</div><div>
<p class=""><span class="">`Erlang OTP 17`</span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_msec', 8640000).</font></span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_sec', 8640).</font></span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p>
<p class=""><span class=""></span><br></p>
<p class=""><span class="">`Erlang OTP 18`</span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_msec', 86400000).</font></span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_sec', 86400).</font></span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 6:00 PM, Sereysethy TOUCH <span dir="ltr"><<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I got it LoÃc.<div><br></div><div>Thanks.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 5:52 PM, Sereysethy TOUCH <span dir="ltr"><<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Something like this?<div><br></div><div>
<p><span><font face="monospace, monospace">{release, {ws_user, "1"},</font></span></p>
<p><span><font face="monospace, monospace">Â Â Â Â [ws_user]}.</font></span></p>
<p><span><font face="monospace, monospace">{sys_config, "./config/sys.config"}.</font></span></p>
<p><span><font face="monospace, monospace">%{paths, ["../deps/"]}.</font></span></p>
<p><span><font face="monospace, monospace">% generate startup script, yes</font></span></p>
<p><span><font face="monospace, monospace">{extended_start_script, true}.</font></span></p></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 5:46 PM, Loïc Hoguin <span dir="ltr"><<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Add {sys_config, "path/to/sys.config"} to your relx.config, then create that sys.config file with the right options and run make.<span><br>
<br>
On 09/21/2015 12:44 PM, Sereysethy TOUCH wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br><span>
You meant _rel or I need to create a folder "rel" and put sys.config inside?<br>
<br>
Sethy<br>
<br>
On Mon, Sep 21, 2015 at 5:09 PM, Loïc Hoguin <<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a><br></span><span>
<mailto:<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a>>> wrote:<br>
<br>
  On 09/21/2015 12:05 PM, Sereysethy TOUCH wrote:<br>
<br>
    Hello,<br>
<br>
    My application is both TLS-client & server. It is a web server which<br>
    allows client browser to connect to server using websocket. I<br>
    use cowboy<br>
    as web server & web socket server.<br>
<br>
    In erlang 17, our application runs for weeks and there is no memory<br>
    issue. We only restart it whenever we have a new update. But<br>
    since we<br>
    update to elang 18, we need to restart our application every 2<br>
    hours.<br>
<br></span>
    I use <a href="http://erlang.mk" rel="noreferrer" target="_blank">erlang.mk</a> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> to compile<span><br>
    our application, I am not<br>
    sure how to set application env. settings. Is it the same if I use<br></span>
    <a href="http://erlang.mk" rel="noreferrer" target="_blank">erlang.mk</a> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>>?<span><br>
<br>
<br>
  If you run a release, you can put the configuration in the<br>
  sys.config file (by default rel/sys.config) and run make again. See<br>
  <a href="http://www.erlang.org/doc/man/config.html" rel="noreferrer" target="_blank">http://www.erlang.org/doc/man/config.html</a><br>
<br>
<br>
<br>
    Thanks,<br>
    Sethy<br>
<br>
    On Mon, Sep 21, 2015 at 4:54 PM, Ingela Andin<br>
    <<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>><br></span>
    <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>>>><span><br>
    wrote:<br>
<br>
       Hi!<br>
<br>
<br>
       2015-09-21 10:09 GMT+02:00 Sereysethy TOUCH<br>
       <<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>>:<br>
<br>
<br>
         Hello,<br>
<br>
         One question, why did my application work fine on<br>
    Erlang 17.5.x<br>
         branch?<br>
<br>
<br>
       Well I am not sure what your application does.<br>
<br>
       If you have a client application that spawns a lot of parallel<br>
       connections to the same host, not waiting for one to<br>
    succeed before<br>
       spawning,<br>
       the client session table could grow a lot. However I have<br>
       implemented a feature to only save unique TLS-sessions in this<br>
       senario to avoid this, and there was a bug so that this<br>
    feature did<br>
       not work, that I have fix in the maint branch. This bug<br>
    however was<br>
       in 17 as well as 18.<br>
<br>
       However the ets info you sent suggest you have a server<br>
    application<br>
       and then setting session_liftime can help keeping the servers<br>
       session table<br>
       smaller. This also is true for clients, that can get a big<br>
    table<br>
       also through connecting to many diffrent servers.<br>
<br>
       Maybe your application can be both a TLS-client and server?<br>
    There is<br>
       no obvious reason from what you told us so far that it<br>
    should work<br>
       better in 17 then 18.<br>
<br>
<br>
         If I want to set this session_lifetime option, where<br>
    should put it?<br>
<br>
<br>
       It is an application environment setting. Search for app in<br>
    the docs.<br>
<br>
<br>
         So there is no use that I should build the erlang 18<br>
    from maint<br>
         branch?<br>
<br>
<br>
<br>
       It is always a good thing to try the latest version, and if you<br>
       still have a problem it would be good if you could try<br>
    providing a<br>
       way to reproduce the problem.<br>
<br>
       Regards Ingela Erlang/OTP tema Ericsson AB<br>
<br>
<br>
         Thanks,<br>
         Sethy<br>
<br>
         On Mon, Sep 21, 2015 at 2:51 PM, Ingela Andin<br>
         <<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>><br></div></div>
    <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>>>><span><br>
    wrote:<br>
<br>
           Hello,<br>
<br>
           2015-09-20 15:24 GMT+02:00 Sereysethy TOUCH<br>
           <<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
           <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>>:<br>
<br>
<br>
             Hello,<br>
<br>
             I did as what you told me by getting a shell<br>
    console on<br>
             the node.<br>
<br>
             I run memory(). and see ets ate the most<br>
    memory. And it<br>
             increased overtime. I run ets:i(). and I found<br>
    out the<br>
             ssl_manager took a big chunk of what memory used.<br>
<br>
             32794      client_ssl_otp_session_cache<br>
    ordered_set<br>
             0   89    ssl_manager<br>
             36891      server_ssl_otp_session_cache<br>
    ordered_set<br>
             1564  67995  ssl_manager<br>
             40993<br>
     httpc_manager__session_cookie_db bag  0<br>
                299   httpc_manager<br>
<br>
             So I think there is a problem with ssl_manager<br>
    in Erlang<br>
             OTP 18.<br>
<br>
             Any workarounds?<br>
<br>
<br>
<br>
           Ok, the bug I fixed with the session table is on<br>
    the client<br>
           side. But your big ets table is on the server side.<br>
           Session data is by default fairly long lived (24 h<br>
    that is<br>
           the max recommended time to keep a session by the RFC),<br>
           you can set the ssl application variable<br>
    session_lifetime to<br>
           make its lifetime shorter which should mitigate<br>
    your problem.<br>
           We have a backlog item to make a configurable<br>
    threshold of<br>
           max sessions allowed to be stored in the table,<br>
    which I am<br>
           sure will be<br>
           implemented fairly soon but alas not for 18.1.<br>
<br>
           Regards Ingela Erlang/OTP Team - Ericsson AB<br>
<br>
             Sethy<br>
<br>
             On Sun, Sep 20, 2015 at 2:22 AM, Jesper Louis<br>
    Andersen<br>
             <<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a><br>
    <mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a>><br></div></div>
             <mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a><span><br>
    <mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a>>>> wrote:<br>
<br>
<br>
               On Sat, Sep 19, 2015 at 6:34 PM, Sereysethy<br>
    TOUCH<br>
               <<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
               <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
    <mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>> wrote:<br>
<br>
                 I just recently updated Erlang to<br>
    latest version<br>
                 OTP 18 on Ubuntu server. It uses cowboy<br>
                 (websocket), ranch, ssl, erlydtl &<br>
    rabbitmq. It<br>
                 used to work fine in OTP 17. The program is<br>
                 correctly compiled but during the<br>
    execution the<br>
                 memory kept increasing. I need to<br>
    restart the<br>
                 process every one or two hours to free<br>
    some memory.<br>
<br>
<br>
               1. Get a shell console on the node<br>
               2. call 'memory().' in the shell<br>
               3. If it reports ets as the winner, call<br>
               4. ets:i().<br>
<br>
               This will give you a nice overview for<br>
    where to look<br>
               and verify it is the SSL problem.<br>
<br>
<br>
               --<br>
               J.<br>
<br>
<br>
<br>
             _______________________________________________<br>
             erlang-questions mailing list<br>
    <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a> <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>><br></div></div>
             <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><span><br>
    <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>>><br>
    <a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
    _______________________________________________<br>
    erlang-questions mailing list<br></span>
    <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a> <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>><span><br>
    <a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
  --<br>
  Loïc Hoguin<br>
  <a href="http://ninenines.eu" rel="noreferrer" target="_blank">http://ninenines.eu</a><br>
  Author of The Erlanger Playbook,<br>
  A book about software development using Erlang<br>
<br>
<br>
</span></blockquote><div><div>
<br>
-- <br>
Loïc Hoguin<br>
<a href="http://ninenines.eu" rel="noreferrer" target="_blank">http://ninenines.eu</a><br>
Author of The Erlanger Playbook,<br>
A book about software development using Erlang<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>