To recover the state, I am also using one of the options given in this
thread. But, to prevent memory starvation (preventing is more efficient
than recovering, in my opinion), one can use erlang:memory() to build a
monitor for the main application. So, a solution would be to have few
"flags" for different levels of memory consumption and to instruct your
application to return a "busy" flag (data not accepted) to the user in
case the memory usage is over a certain level. For more complex
monitors, one can use os:cmd() to get the system resources consumption.
Another solution is to save the state only when the memory consumption
is above a threshold, so, in case of crash, to be able to start from the
last known state. This is less time consuming (to save the state every
time it got changed is introducing a lag in your application), but you
need to take into consideration many factors to decrease the risk of
losing the state.<br>
<br>I hope this will help you to have at least a starting point in (re)designing your application.<div class="yj6qo ajU"><div id=":1le" class="ajR" tabindex="0"><img class="ajT" src="images/cleardot.gif"><br><br>CGS<br></div>
</div><br><br><div class="gmail_quote">On Tue, Jan 17, 2012 at 10:19 PM, Jesper Louis Andersen <span dir="ltr"><<a href="mailto:jesper.louis.andersen@erlang-solutions.com">jesper.louis.andersen@erlang-solutions.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="im">
On 1/16/12 10:52 PM, Roman Shestakov wrote:
<blockquote type="cite">
<div style="font-size:12pt;font-family:times new roman,new york,times,serif">
<div style="font-family:times new roman,new york,times,serif;font-size:12pt">
<div style="font-family:times new roman,new york,times,serif;font-size:12pt"><br>
<div>
<div>
<div style="font-size:12pt;font-family:times new roman,new york,times,serif">
<div>what is the correct way to recover "stateful"
Erlang application? In my case, the app. which is
crashing is a complex hierarchy of fsm_processes
each containing certain state. I understand how to
recover stateless processes with supervisors but
what is the correct way to recovery stateful apps?
Clearly in my case I probably need some kind of
supervisor 'node' but what would be the steps to
correctly recover killed processes with their
states? do I need to use a db and replay the
processes from disk on another node or can I have a
node with identical processes hierarchy?<br>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote></div>
The problem with a crashing process is that its internal state is
not sound anymore. There was a reason as to why it went wrong. The
problem with a crashing node is largely the same. There is a reason
you ended up with resource exhaustion in the first place.<br>
<br>
The trick is that there is no trick. You need another node to have
your state or you need your state on stable storage once in a while
so you can restart from it. The point is that you can then make sure
that from this stable state there will be no trouble. Essentially
you want to only store to disk when you are sure about some part of
the system is consistent with your invariants. Or move your state to
another node.<span class="HOEnZb"><font color="#888888"><br>
<pre cols="72">--
Jesper Louis Andersen
Erlang Solutions Ltd., Copenhagen, DK</pre>
</font></span></div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br>