<div dir="ltr">Dear all,<div><br></div><div>Finally I know what is the difference between Erlang 17 and Erlang 18 in ssl module which caused a problem of memory. It is not really a memory leak but simply the caching problem.</div><div><br></div><div>I take a look at the source code of ssl module (ssl_manager.erl) in OTP-17 & OTP-18. It is how they define the 24h in seconds. In OTP-17, somebody did a wrong multiplication. While in OTP-18, the calculations were correct for both 24h-in-sec & 24h-in-msec!</div><div><br></div><div>By the way, thanks to Ingela, Loïc and Jesper for helping me finding the problems and solution.</div><div><br></div><div>Thanks,</div><div>T.</div><div>
<p class=""><span class="">`Erlang OTP 17`</span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_msec', 8640000).</font></span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_sec', 8640).</font></span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p>
<p class=""><span class=""></span><br></p>
<p class=""><span class="">`Erlang OTP 18`</span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_msec', 86400000).</font></span></p>
<p class=""><span class=""><font color="#ff0000">-define('24H_in_sec', 86400).</font></span></p>
<p class=""><span class="">-define(GEN_UNIQUE_ID_MAX_TRIES, 10).</span></p>
<p class=""><span class="">-define(SESSION_VALIDATION_INTERVAL, 60000).</span></p>
<p class=""><span class="">-define(CLEAR_PEM_CACHE, 120000).</span></p>
<p class=""><span class="">-define(CLEAN_SESSION_DB, 60000).</span></p>
<p class=""><span class="">-define(CLEAN_CERT_DB, 500).</span></p>
<p class=""><span class="">-define(NOT_TO_BIG, 10).</span></p></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 6:00 PM, Sereysethy TOUCH <span dir="ltr"><<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I got it LoÏc.<div><br></div><div>Thanks.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 5:52 PM, Sereysethy TOUCH <span dir="ltr"><<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Something like this?<div><br></div><div>
<p><span><font face="monospace, monospace">{release, {ws_user, "1"},</font></span></p>
<p><span><font face="monospace, monospace"> [ws_user]}.</font></span></p>
<p><span><font face="monospace, monospace">{sys_config, "./config/sys.config"}.</font></span></p>
<p><span><font face="monospace, monospace">%{paths, ["../deps/"]}.</font></span></p>
<p><span><font face="monospace, monospace">% generate startup script, yes</font></span></p>
<p><span><font face="monospace, monospace">{extended_start_script, true}.</font></span></p></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 21, 2015 at 5:46 PM, Loïc Hoguin <span dir="ltr"><<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Add {sys_config, "path/to/sys.config"} to your relx.config, then create that sys.config file with the right options and run make.<span><br>
<br>
On 09/21/2015 12:44 PM, Sereysethy TOUCH wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br><span>
You meant _rel or I need to create a folder "rel" and put sys.config inside?<br>
<br>
Sethy<br>
<br>
On Mon, Sep 21, 2015 at 5:09 PM, Loïc Hoguin <<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a><br></span><span>
<mailto:<a href="mailto:essen@ninenines.eu" target="_blank">essen@ninenines.eu</a>>> wrote:<br>
<br>
On 09/21/2015 12:05 PM, Sereysethy TOUCH wrote:<br>
<br>
Hello,<br>
<br>
My application is both TLS-client & server. It is a web server which<br>
allows client browser to connect to server using websocket. I<br>
use cowboy<br>
as web server & web socket server.<br>
<br>
In erlang 17, our application runs for weeks and there is no memory<br>
issue. We only restart it whenever we have a new update. But<br>
since we<br>
update to elang 18, we need to restart our application every 2<br>
hours.<br>
<br></span>
I use <a href="http://erlang.mk" rel="noreferrer" target="_blank">erlang.mk</a> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> to compile<span><br>
our application, I am not<br>
sure how to set application env. settings. Is it the same if I use<br></span>
<a href="http://erlang.mk" rel="noreferrer" target="_blank">erlang.mk</a> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>> <<a href="http://erlang.mk" rel="noreferrer" target="_blank">http://erlang.mk</a>>?<span><br>
<br>
<br>
If you run a release, you can put the configuration in the<br>
sys.config file (by default rel/sys.config) and run make again. See<br>
<a href="http://www.erlang.org/doc/man/config.html" rel="noreferrer" target="_blank">http://www.erlang.org/doc/man/config.html</a><br>
<br>
<br>
<br>
Thanks,<br>
Sethy<br>
<br>
On Mon, Sep 21, 2015 at 4:54 PM, Ingela Andin<br>
<<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>><br></span>
<mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>>>><span><br>
wrote:<br>
<br>
Hi!<br>
<br>
<br>
2015-09-21 10:09 GMT+02:00 Sereysethy TOUCH<br>
<<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>>:<br>
<br>
<br>
Hello,<br>
<br>
One question, why did my application work fine on<br>
Erlang 17.5.x<br>
branch?<br>
<br>
<br>
Well I am not sure what your application does.<br>
<br>
If you have a client application that spawns a lot of parallel<br>
connections to the same host, not waiting for one to<br>
succeed before<br>
spawning,<br>
the client session table could grow a lot. However I have<br>
implemented a feature to only save unique TLS-sessions in this<br>
senario to avoid this, and there was a bug so that this<br>
feature did<br>
not work, that I have fix in the maint branch. This bug<br>
however was<br>
in 17 as well as 18.<br>
<br>
However the ets info you sent suggest you have a server<br>
application<br>
and then setting session_liftime can help keeping the servers<br>
session table<br>
smaller. This also is true for clients, that can get a big<br>
table<br>
also through connecting to many diffrent servers.<br>
<br>
Maybe your application can be both a TLS-client and server?<br>
There is<br>
no obvious reason from what you told us so far that it<br>
should work<br>
better in 17 then 18.<br>
<br>
<br>
If I want to set this session_lifetime option, where<br>
should put it?<br>
<br>
<br>
It is an application environment setting. Search for app in<br>
the docs.<br>
<br>
<br>
So there is no use that I should build the erlang 18<br>
from maint<br>
branch?<br>
<br>
<br>
<br>
It is always a good thing to try the latest version, and if you<br>
still have a problem it would be good if you could try<br>
providing a<br>
way to reproduce the problem.<br>
<br>
Regards Ingela Erlang/OTP tema Ericsson AB<br>
<br>
<br>
Thanks,<br>
Sethy<br>
<br>
On Mon, Sep 21, 2015 at 2:51 PM, Ingela Andin<br>
<<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>><br></div></div>
<mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a> <mailto:<a href="mailto:ingela.andin@gmail.com" target="_blank">ingela.andin@gmail.com</a>>>><span><br>
wrote:<br>
<br>
Hello,<br>
<br>
2015-09-20 15:24 GMT+02:00 Sereysethy TOUCH<br>
<<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>>:<br>
<br>
<br>
Hello,<br>
<br>
I did as what you told me by getting a shell<br>
console on<br>
the node.<br>
<br>
I run memory(). and see ets ate the most<br>
memory. And it<br>
increased overtime. I run ets:i(). and I found<br>
out the<br>
ssl_manager took a big chunk of what memory used.<br>
<br>
32794 client_ssl_otp_session_cache<br>
ordered_set<br>
0 89 ssl_manager<br>
36891 server_ssl_otp_session_cache<br>
ordered_set<br>
1564 67995 ssl_manager<br>
40993<br>
httpc_manager__session_cookie_db bag 0<br>
299 httpc_manager<br>
<br>
So I think there is a problem with ssl_manager<br>
in Erlang<br>
OTP 18.<br>
<br>
Any workarounds?<br>
<br>
<br>
<br>
Ok, the bug I fixed with the session table is on<br>
the client<br>
side. But your big ets table is on the server side.<br>
Session data is by default fairly long lived (24 h<br>
that is<br>
the max recommended time to keep a session by the RFC),<br>
you can set the ssl application variable<br>
session_lifetime to<br>
make its lifetime shorter which should mitigate<br>
your problem.<br>
We have a backlog item to make a configurable<br>
threshold of<br>
max sessions allowed to be stored in the table,<br>
which I am<br>
sure will be<br>
implemented fairly soon but alas not for 18.1.<br>
<br>
Regards Ingela Erlang/OTP Team - Ericsson AB<br>
<br>
Sethy<br>
<br>
On Sun, Sep 20, 2015 at 2:22 AM, Jesper Louis<br>
Andersen<br>
<<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a><br>
<mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a>><br></div></div>
<mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a><span><br>
<mailto:<a href="mailto:jesper.louis.andersen@gmail.com" target="_blank">jesper.louis.andersen@gmail.com</a>>>> wrote:<br>
<br>
<br>
On Sat, Sep 19, 2015 at 6:34 PM, Sereysethy<br>
TOUCH<br>
<<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>><br></span>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a><div><div><br>
<mailto:<a href="mailto:touch.sereysethy@gmail.com" target="_blank">touch.sereysethy@gmail.com</a>>>> wrote:<br>
<br>
I just recently updated Erlang to<br>
latest version<br>
OTP 18 on Ubuntu server. It uses cowboy<br>
(websocket), ranch, ssl, erlydtl &<br>
rabbitmq. It<br>
used to work fine in OTP 17. The program is<br>
correctly compiled but during the<br>
execution the<br>
memory kept increasing. I need to<br>
restart the<br>
process every one or two hours to free<br>
some memory.<br>
<br>
<br>
1. Get a shell console on the node<br>
2. call 'memory().' in the shell<br>
3. If it reports ets as the winner, call<br>
4. ets:i().<br>
<br>
This will give you a nice overview for<br>
where to look<br>
and verify it is the SSL problem.<br>
<br>
<br>
--<br>
J.<br>
<br>
<br>
<br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a> <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>><br></div></div>
<mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><span><br>
<mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>>><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
erlang-questions mailing list<br></span>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a> <mailto:<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>><span><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
--<br>
Loïc Hoguin<br>
<a href="http://ninenines.eu" rel="noreferrer" target="_blank">http://ninenines.eu</a><br>
Author of The Erlanger Playbook,<br>
A book about software development using Erlang<br>
<br>
<br>
</span></blockquote><div><div>
<br>
-- <br>
Loïc Hoguin<br>
<a href="http://ninenines.eu" rel="noreferrer" target="_blank">http://ninenines.eu</a><br>
Author of The Erlanger Playbook,<br>
A book about software development using Erlang<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>