<div dir="ltr">Akash,<div><br></div><div>First step of curing this is finding out what is eating the memory. Data on the sockets, opened files, in-memory database, atom storage, in-memory data structure, number of Erlang processes spawned with each using considerable amount of memory... there are plenty more. If you figure out what is eating the memory, you can put a CAP on that specific thing, than just limiting the total memory available to the VM/OS process. I would suggest to find out the root cause and revisit the design.</div><div><br></div><div>Theepan</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 5, 2015 at 5:37 PM, Akash Chowdhury <span dir="ltr"><<a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The use case is that the application is using huge memory. In some occurrences, one node grew until 100GB and the application as well as the box crashed. We would like to setup a limit so that application and box doesn't crash.<div>Thanks.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Sep 5, 2015 at 4:20 AM, Michael Truog <span dir="ltr"><<a href="mailto:mjtruog@gmail.com" target="_blank">mjtruog@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000"><span>
    <div>On 09/04/2015 06:44 PM, Zulfiqer
      Sekender wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">Thanks for the reply. Can't we do anything from
        Erlang level?<br>
        Thanks.<br>
      </div>
    </blockquote></span>
    Its a bit complex to track the exact amount of memory inside the
    Erlang VM in Erlang source code to do<br>
    something useful with the information due to the various memory
    pools and the garbage collection.<br>
    However, there is some code at
    <a href="https://github.com/okeuday/erlang_term/" target="_blank">https://github.com/okeuday/erlang_term/</a> which returns the upper
    limit<br>
    for some Erlang term's memory consumption.  The result can then be
    used in Erlang source code for<br>
    determining how to avoid excessive memory consumption, to avoid a
    crash.  CloudI uses this to limit the<br>
    incoming queue of a CloudI service with the queue_size service
    configuration option<br>
    (<a href="http://cloudi.org/api.html#2_services_add_config_opts" target="_blank">http://cloudi.org/api.html#2_services_add_config_opts</a>).<div><div><br>
    <br>
    <blockquote type="cite">
      <div dir="ltr"><br>
        <hr>Date: Sat, 5 Sep 2015 05:45:01 +0530<br>
        From: <a href="mailto:vasdeveloper@gmail.com" target="_blank">vasdeveloper@gmail.com</a><br>
        To: <a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a><br>
        CC: <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
        Subject: Re: [erlang-questions] Setting maximum amount of data
        allocated by an Erlang node.‏<br>
        <br>
        <div dir="ltr">Please look at OS level configurations. If you
          are using Linux or Unix system, check ULIMIT command.
          <div><br>
          </div>
          <div>Theepan</div>
        </div>
        <div><br>
          <div>On Sat, Sep 5, 2015 at 5:23 AM,
            Akash Chowdhury <span dir="ltr"><<a href="mailto:achowdhury918@gmail.com" target="_blank">achowdhury918@gmail.com</a>></span>
            wrote:<br>
            <blockquote style="border-left:1px #ccc solid;padding-left:1ex">
              <div dir="ltr">
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">Hi,</p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">I
                  would like to set a number for</p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px">a)<span>      </span></span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">simultaneously
                    alive maximum Erlang processes in an Erlang node
                     and</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px">b)<span>      </span></span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">for
                    maximum amount of data allocated by an Erlang node</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">I
                    know that for process, I have to use the<span style="line-height:20.8267px"> </span><span style="line-height:20.8267px"><b style="line-height:20.8267px"><a href="http://www.erlang.org/doc/man/erl.html#max_processes" style="line-height:20.8267px;font-weight:inherit;color:blue" target="_blank"><span style="line-height:20.8267px"><span style="line-height:20.8267px;color:windowtext;font-weight:normal;text-decoration:none">+P</span></span></a></b></span><span style="line-height:20.8267px"> </span>command-line
                    flag. But what I have to do for setting maximum
                    amount of data allocated by an Erlang node.</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial">I
                    looked at the following documentation:</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);text-indent:0.5in"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial"><a href="http://www.erlang.org/doc/efficiency_guide/advanced.html" style="line-height:20.8267px;font-weight:inherit;color:blue" target="_blank">http://www.erlang.org/doc/efficiency_guide/advanced.html</a></span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial">It
                    says setting </span><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">maximum
                    amount of data allocated by an Erlang node is
                    possible, but it doesn’t say how.</span><span style="line-height:20.8267px;color:rgb(31,73,125);background-image:initial;background-repeat:initial"></span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Any
                    help will be highly appreciated.</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Thanks.</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">-</span></p>
                <p style="line-height:20.8267px;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><span style="line-height:20.8267px;background-image:initial;background-repeat:initial">Akash</span></p>
              </div>
              <br>
              _______________________________________________<br>
              erlang-questions mailing list<br>
              <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
              <a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
              <br>
            </blockquote>
          </div>
          <br>
        </div>
        <br>
        _______________________________________________
        erlang-questions mailing list
        <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>
        <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a> </div>
      <br>
      <fieldset></fieldset>
      <br>
      <pre>_______________________________________________
erlang-questions mailing list
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
    </blockquote>
    <br>
  </div></div></div>

<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>