<div dir="ltr">Akash,<div><br></div><div>First step of curing this is finding out what is eating the memory. Data on the sockets, opened files, in-memory database, atom storage, in-memory data structure, number of Erlang processes spawned with each using considerable amount of memory... there are plenty more. If you figure out what is eating the memory, you can put a CAP on that specific thing, than just limiting the total memory available to the VM/OS process. I would suggest to find out the root cause and revisit the design.</div><div><br></div><div>Theepan</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 5, 2015 at 5:37 PM, Akash Chowdhury <span dir="ltr"><<a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The use case is that the application is using huge memory. In some occurrences, one node grew until 100GB and the application as well as the box crashed. We would like to setup a limit so that application and box doesn't crash.<div>Thanks.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 5, 2015 at 4:20 AM, Michael Truog <span dir="ltr"><<a href="mailto:mjtruog@gmail.com" target="_blank">mjtruog@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span>
<div>On 09/04/2015 06:44 PM, Zulfiqer
Sekender wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks for the reply. Can't we do anything from
Erlang level?<br>
Thanks.<br>
</div>
</blockquote></span>
Its a bit complex to track the exact amount of memory inside the
Erlang VM in Erlang source code to do<br>
something useful with the information due to the various memory
pools and the garbage collection.<br>
However, there is some code at
<a href="https://github.com/okeuday/erlang_term/" target="_blank">https://github.com/okeuday/erlang_term/</a> which returns the upper
limit<br>
for some Erlang term's memory consumption. The result can then be
used in Erlang source code for<br>
determining how to avoid excessive memory consumption, to avoid a
crash. CloudI uses this to limit the<br>
incoming queue of a CloudI service with the queue_size service
configuration option<br>
(<a href="http://cloudi.org/api.html#2_services_add_config_opts" target="_blank">http://cloudi.org/api.html#2_services_add_config_opts</a>).<div><div><br>
<br>
<blockquote type="cite">
<div dir="ltr"><br>
<hr>Date: Sat, 5 Sep 2015 05:45:01 +0530<br>
From: <a href="mailto:vasdeveloper@gmail.com" target="_blank">vasdeveloper@gmail.com</a><br>
To: <a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a><br>
CC: <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
Subject: Re: [erlang-questions] Setting maximum amount of data
allocated by an Erlang node.<br>
<br>
<div dir="ltr">Please look at OS level configurations. If you
are using Linux or Unix system, check ULIMIT command.
<div><br>
</div>
<div>Theepan</div>
</div>
<div><br>
<div>On Sat, Sep 5, 2015 at 5:23 AM,
Akash Chowdhury <span dir="ltr"><<a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a>></span>
wrote:<br>
<blockquote style="border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">Hi,</p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">I
would like to set a number for</p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px">a)<span> </span></span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">simultaneously
alive maximum Erlang processes in an Erlang node
and</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px">b)<span> </span></span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">for
maximum amount of data allocated by an Erlang node</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">I
know that for process, I have to use the<span style="line-height:20.8267px"> </span><span style="line-height:20.8267px"><b style="line-height:20.8267px"><a href="http://www.erlang.org/doc/man/erl.html#max_processes" style="line-height:20.8267px;font-weight:inherit;color:blue" target="_blank"><span style="line-height:20.8267px"><span style="line-height:20.8267px;color:windowtext;font-weight:normal;text-decoration:none">+P</span></span></a></b></span><span style="line-height:20.8267px"> </span>command-line
flag. But what I have to do for setting maximum
amount of data allocated by an Erlang node.</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial">I
looked at the following documentation:</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);text-indent:0.5in"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial"><a href="http://www.erlang.org/doc/efficiency_guide/advanced.html" style="line-height:20.8267px;font-weight:inherit;color:blue" target="_blank">http://www.erlang.org/doc/efficiency_guide/advanced.html</a></span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial">It
says setting </span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">maximum
amount of data allocated by an Erlang node is
possible, but it doesn’t say how.</span><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial"></span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Any
help will be highly appreciated.</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Thanks.</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">-</span></p>
<p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Akash</span></p>
</div>
<br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
_______________________________________________
erlang-questions mailing list
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a> </div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
erlang-questions mailing list
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>