[erlang-questions] [eeps] New EEP: setrlimit(2) analogue for Erlang
Thu Feb 7 20:12:14 CET 2013
On Thu, Feb 7, 2013 at 7:27 AM, Björn-Egil Dahlberg <egil@REDACTED> wrote:
> I dug out what I wrote a year ago ..
> Reference implementation:
> Remember, this is a prototype and a reference implementation.
> There is a couple of issues not addressed or at least open-ended.
Looks great! I truly hope this will get accepted rather sooner than
later, at least as an experimental feature.
Does the proposal cover refc binaries as well? Almost all interesting
use cases that I can imagine for limits involve binaries.
Here's one test case:
1. Create a web server in Erlang e.g. with Mochiweb, Cowboy, Inets.
2. Create a request handler that expects to receive Zip files, which
it extracts with zip:extract(Request, [memory]).
3. Create a zip bomb : dd if=/dev/zero bs=1M count=8000 | zip req.zip -
4. POST the small req.zip to the web server.
5. See the VM go down in flames.
Obviously the per-process limits would elegantly solve this problem if
they covered binaries as well. A far less elegant solution would be to
handle the unsafe decompression with an external process and open_port
(inefficient) or implement a sophisticated alternative to the zip
module which handles the limits by itself (inefficient, annoying).
I understand that limiting the message queue / ets / NIFs can be
trickier. Just covering the basic max-heap with binaries would be a
good starting point.
> * Should processes be able to set limits on other processes? I think not
> though my draft argues for it. It introduces unnecessary restraints on erts
> and hinders performance. 'save_calls' is such an option.
> * ets - if your table increases beyond some limit. Who should we punish? The
> inserter? The owner? What would be the rationale? We cannot just punish the
> inserter, the ets table is still there taking a lot of memory and no other
> process could insert into the table. They would be killed as well. Remove
> the owner and hence the table (and potential heir)? What kind of problems
> would arise then? Limits should be tied into a supervision strategy and
> restart the whole thing.
> * In my draft and reference implementation I use soft limits. Once a process
> reaches its limit it will be marked for termination by an exit signal. The
> trouble here is there is no real guarantee for how long this will take. A
> process can continue appending a binary for a short while and ending the
> beam with OOM still. (If I remember it correctly you have to schedule out to
> terminate a process in SMP thus you need to bump all reduction. But, not all
> things handle return values from the garbage collector, most notably within
> the append_binary instruction). There may be other issues as well.
> * Message queues. In the current implementation of message queues we have
> two queues. An inner one which is locked by the receiver process while
> executing and an outer one which other processes will use and thus not
> compete for a message queue lock with the executing process. When the inner
> queue is depleted the receiver process will lock the outer queue and move
> the entire thing to the inner one. Rinse and repeat. The only guarantee we
> have to ensure with our implementation is: signal order between two
> processes. So, in the future we might have several queues to improve
> performance. If you introduce monitoring of the total number messages in the
> abstracted queue (all the queues) this will most probable kill any sort of
> scalability. For instance a sender would not be allowed to check the inner
> queue for this reason. Would a "fast" counter check in the inner queue be
> allowed? Perhaps if it is fast enough, but any sort of bookkeeping costs
> performance. If we introduce even more queues for scalability reasons this
> will cost even more.
> * What about other memory users? Drivers? NIFs?
> I do believe in increments in development as long it is path to the
> envisioned goal.
> And to reiterate, i'm not convinced that limits on just processes is the way
> to go. I think a complete monitoring system should be envisioned, not just
> for processes.
> // Björn-Egil
> On 2013-02-06 23:03, Richard O'Keefe wrote:
> Just today, I saw Matthew Evans'
> This pertains to a feature I would like to see
> in Erlang. The ability to set an optional
> "memory limit" when a process and ETS table is
> created (and maybe a global optional per-process
> limit when the VM is started). I've seen a few
> cases where, due to software bugs, a process size
> grows and grows; unfortunately as things stand
> today the result is your entire VM crashing -
> hopefully leaving you with a crash_dump.
> Having such a limit could cause the process to
> terminate (producing a OOM crash report in
> erlang.log) and the crashing process could be
> handled with supervisor rules. Even better you
> can envisage setting the limits artificially low
> during testing to catch these types of bugs early on.
> in my mailbox. I have seen too many such e-mail messages.
> Here's a specific proposal. It's time _something_ was done
> about this kind of problem. I don't expect that my EEP is
> the best way to deal with it, but at least there's going to
> be something for people to point to.
> eeps mailing list
> erlang-questions mailing list
More information about the erlang-questions