<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">For what it is worth.<div>I located my old implementation (three years old :-)<div><br></div><div><a href="https://github.com/tonyrog/otp/tree/limits/">https://github.com/tonyrog/otp/tree/limits/</a></div><div><br><div>A very brief description (I presented some early stuff for OTP around then)</div><div>I do not think that max_message_queue_len is implemented,</div><div>not really defined. Too many options here.</div><div><br></div><div>/Tony</div><div><br></div><div><div>Resource Limits in Erlang</div><div>=========================</div><div><br></div><div># Why?</div><div><br></div><div>- Ability to detect and kill runaway processes. </div><div>- Detect and kill zombies.</div><div>- Basis of safe mode framework.</div><div>- Excellent to use in debugging.</div><div><br></div><div># How?</div><div><br></div><div>- Limits are checked at context switch and garbage collection time. </div><div>- Relatively light weight.</div><div>- Create limits by using spawn_opt.</div><div>- Limits are inherited by spawned processes.</div><div>- If a limit is reached a signal 'system_limit' is raised.</div><div><br></div><div>## max_memory</div><div>Limit the amount of memory a process may use. Account for all memory a </div><div>process is using at any given time. This includes heap, stack, tables, </div><div>links and messages. The memory is shared among all processes spawned by the </div><div>process that where limited. spawn\_opt can be used to give </div><div>away some of that memory to child processes.</div><div><br></div><div>## max\_time</div><div>Controls how many milliseconds a process may use in "wall clock" time.</div><div>Created (sub-)processes will only be able to run for the remaining time.</div><div><br></div><div>## max\_cpu</div><div>Controls how many milliseconds a process may run in cpu time. </div><div>The cpu time is consumed by the process and all it’s spawned (sub-)processes.</div><div><br></div><div>## max\_reductions</div><div>A lighter version of max\_cpu. One reductions does approximately corresponds</div><div>to a function call.</div><div><br></div><div>## max\_processes</div><div>Limit number of running (sub-)processes that may be running at any given time.</div><div><br></div><div>## max\_ports</div><div>Limit number of open ports that can be open at any given time.</div><div><br></div><div>## max\_tables</div><div>Limit number of ets tables that may be open at any given time.</div><div><br></div><div>## max\_message\_queue\_len</div><div>Limit the size of the message queue. Who dies when the limit is reached?</div><div>Either sender or receiver? Maybe add a dangerous block option?</div><div><br></div><div># process\_info</div><div><br></div><div>process\_info is used to read the current limits and the</div><div>"remaining" quota. </div><div>process_info(Pid, max\_cpu) is used to read the number of</div><div>milliseconds set for execution while process_info(Pid, remaining\_cpu)</div><div>return how many cpu milliseconds that remain to execute.</div><div>The items include: max\_process, max\_ports, max\_tables, max\_memory, </div><div>max\_reductions, max\_message\_queue\_len, max\_cpu, max\_time,</div><div>remaining\_process, remaining\_ports, remaining_tables, remaining\_memory, </div><div>remaining\_reductions, remaining\_message\_queue\_len, </div><div>remaining\_cpu, remaining\_time</div></div><div><br></div><div><br><div><div>On 7 feb 2013, at 16:27, Björn-Egil Dahlberg <<a href="mailto:egil@erlang.org">egil@erlang.org</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">I dug out what I wrote a year ago ..<br>
<br>
eep-draft:<br>
<a class="moz-txt-link-freetext" href="https://github.com/psyeugenic/eep/blob/egil/system_limits/eeps/eep-00xx.md">https://github.com/psyeugenic/eep/blob/egil/system_limits/eeps/eep-00xx.md</a><br>
<br>
Reference implementation:<br>
<a class="moz-txt-link-freetext" href="https://github.com/psyeugenic/otp/commits/egil/limits-system-gc/OTP-9856">https://github.com/psyeugenic/otp/commits/egil/limits-system-gc/OTP-9856</a><br>
Remember, this is a prototype and a reference implementation.<br>
<br>
There is a couple of issues not addressed or at least open-ended.<br>
<br>
* Should processes be able to set limits on other processes? I
think not though my draft argues for it. It introduces unnecessary
restraints on erts and hinders performance. 'save_calls' is such
an option.<br>
<br>
* ets - if your table increases beyond some limit. Who should we
punish? The inserter? The owner? What would be the rationale? We
cannot just punish the inserter, the ets table is still there
taking a lot of memory and no other process could insert into the
table. They would be killed as well. Remove the owner and hence
the table (and potential heir)? What kind of problems would arise
then? Limits should be tied into a supervision strategy and
restart the whole thing.<br>
<br>
* In my draft and reference implementation I use soft limits. Once
a process reaches its limit it will be marked for termination by
an exit signal. The trouble here is there is no real guarantee for
how long this will take. A process can continue appending a binary
for a short while and ending the beam with OOM still. (If I
remember it correctly you have to schedule out to terminate a
process in SMP thus you need to bump all reduction. But, not all
things handle return values from the garbage collector, most
notably within the append_binary instruction). There may be other
issues as well.<br>
<br>
* Message queues. In the current implementation of message queues
we have two queues. An inner one which is locked by the receiver
process while executing and an outer one which other processes
will use and thus not compete for a message queue lock with the
executing process. When the inner queue is depleted the receiver
process will lock the outer queue and move the entire thing to the
inner one. Rinse and repeat. The only guarantee we have to ensure
with our implementation is: signal order between two processes.
So, in the future we might have several queues to improve
performance. If you introduce monitoring of the total number
messages in the abstracted queue (all the queues) this will most
probable kill any sort of scalability. For instance a sender would
not be allowed to check the inner queue for this reason. Would a
"fast" counter check in the inner queue be allowed? Perhaps if it
is fast enough, but any sort of bookkeeping costs performance. If
we introduce even more queues for scalability reasons this will
cost even more.<br>
<br>
* What about other memory users? Drivers? NIFs?<br>
<br>
I do believe in increments in development as long it is path to
the envisioned goal.<br>
And to reiterate, i'm not convinced that limits on just processes
is the way to go. I think a complete monitoring system should be
envisioned, not just for processes.<br>
<br>
// Björn-Egil<br>
<br>
On 2013-02-06 23:03, Richard O'Keefe wrote:<br>
</div>
<blockquote cite="mid:85819AB4-AD04-401F-B814-88157B71BE07@cs.otago.ac.nz" type="cite">
<pre wrap="">Just today, I saw Matthew Evans'
This pertains to a feature I would like to see
in Erlang. The ability to set an optional
"memory limit" when a process and ETS table is
created (and maybe a global optional per-process
limit when the VM is started). I've seen a few
cases where, due to software bugs, a process size
grows and grows; unfortunately as things stand
today the result is your entire VM crashing -
hopefully leaving you with a crash_dump.
Having such a limit could cause the process to
terminate (producing a OOM crash report in
erlang.log) and the crashing process could be
handled with supervisor rules. Even better you
can envisage setting the limits artificially low
during testing to catch these types of bugs early on.
in my mailbox. I have seen too many such e-mail messages.
Here's a specific proposal. It's time _something_ was done
about this kind of problem. I don't expect that my EEP is
the best way to deal with it, but at least there's going to
be something for people to point to.
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
eeps mailing list
<a class="moz-txt-link-abbreviated" href="mailto:eeps@erlang.org">eeps@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/eeps">http://erlang.org/mailman/listinfo/eeps</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>http://erlang.org/mailman/listinfo/erlang-questions<br></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px; "><div><span class="Apple-style-span" style="color: rgb(51, 51, 51); font-family: Geneva, Arial, Helvetica, sans-serif; font-size: 12px; ">"Installing applications can lead to corruption over time. </span><span class="Apple-style-span" style="color: rgb(51, 51, 51); font-family: Geneva, Arial, Helvetica, sans-serif; font-size: 12px; ">Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix"</span></div><div><span class="Apple-style-span" style="color: rgb(51, 51, 51); font-family: Geneva, Arial, Helvetica, sans-serif; font-size: 12px; "><br></span></div></span><br class="Apple-interchange-newline">
</div>
<br></div></div></div></body></html>