<br><br><div class="gmail_quote">2013/2/5 Matthew Evans <span dir="ltr"><<a href="mailto:mattevans123@hotmail.com" target="_blank">mattevans123@hotmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">
Hi,<div><br></div><div>I don't wish to thread-jack, but I am anyway ;-)</div><div><br></div><div>This pertains to a feature I would like to see in Erlang. The ability to set an optional "memory limit" when a process and ETS table is created (and maybe a global optional per-process limit when the VM is started). I've seen a few cases where, due to software bugs, a process size grows and grows; unfortunately as things stand today the result is your entire VM crashing - hopefully leaving you with a crash_dump. </div>
<div><br></div><div>Having such a limit could cause the process to terminate (producing a OOM crash report in erlang.log) and the crashing process could be handled with supervisor rules. Even better you can envisage setting the limits artificially low during testing to catch these types of bugs early on.</div>
</div></div></blockquote><div><br></div><div>Not this again.</div><div><br></div><div>This idea is not an uninteresting one. A safeguard against bad flow control in application design.</div><div><br></div><div>We, well I, implemented a "limits" prototype where I could set max heap sizes on processes. The process would be killed with an exception limit error if it ever reached its max heap size. It worked fine in most cases but if I remember correctly we couldn't guard against binary creation fast enough (without killing performance). I'm pretty sure we could remedy those problems though, but by then my work queue was reprioritized. </div>
<div><br></div><div>We ultimately decided against it, citing bad application design as exhibit A and root cause to runaway memory consumption; thus not strictly needed. =) </div><div>I think the biggest advantages of limits is like you say, catching problems early in design. </div>
<div>However, I think you can simulate some of this already by using a server process with:</div><div><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace"><br></font></span></div><div><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace"> erlang:system_monitor(MonitorPid, [{large_heap, HeapSize}]),</font></span></div>
<div><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace"> ...</font></span></div><div><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace"> receive</font></span></div>
<div><span style="font-family:Courier,monospace;background-color:rgb(255,255,255)"> </span><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace">{monitor, GcPid, large_heap, _} -> purge_with_fire(GcPid);</font></span></div>
<div><span style="background-color:rgb(255,255,255)"><font face="courier new, monospace"> ...</font></span></div><div><br></div><div>That might suffice for developing purposes.</div><div><br></div><div>If this should be considered again something more extensive then just "limit heaps on processes and ets" should be considered I think. I would like a subscriber system on erts where a could subscribe to different part of the system and handle notifications as I see fit. The trouble here would be "how fast do I need to handle this?". Do a message suffice or do I need to wrap the notification in a fun like a callback?</div>
<div><br></div><div>Perhaps it has som validity to do something simple instead.</div><div><br></div><div>If a good enough proposal is put forward it will be considered.</div><div><br></div><div>// Björn-Egil</div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div dir="ltr"><div><br></div><div>Cheers</div><div><br></div><div>Matt</div><div><br><div><div></div><hr>Date: Tue, 5 Feb 2013 08:51:16 +0100<br>
From: <a href="mailto:gustav.simonsson@gmail.com" target="_blank">gustav.simonsson@gmail.com</a><br>To: <a href="mailto:solomon.wzs@gmail.com" target="_blank">solomon.wzs@gmail.com</a><br>CC: <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
Subject: Re: [erlang-questions] How to see which processes used ets table?<div><div class="h5"><br><br><div dir="ltr"><div><div>Note that erlang:memory/1 gives memory in bytes, whereas ets:i/0 gives it in words. On a 64-bit system 625M words would be 5GB, so the numbers you get might be correct. Try summing the memory from ets:i/0 up exact and compare it to erlang:memory(ets).<br>
<br></div>Cheers,<br></div>Gustav Simonsson<br></div><div><br><br><div>On Tue, Feb 5, 2013 at 3:42 AM, Solomon <span dir="ltr"><<a href="mailto:solomon.wzs@gmail.com" target="_blank">solomon.wzs@gmail.com</a>></span> wrote:<br>
<blockquote style="border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>I checked system info with erlang:memory/1 and found the total amount of memory allocated for ets
tables was about 5GB.<br><br></div>Then I sum the memory size return by ets:i/0 and it was about 600MB.<br><br></div>So which processes used ets table and did not return by ets:i/0?<br></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________
erlang-questions mailing list
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a></div></div></div></div> </div></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br>