<div>
Siri,
</div><div><br></div><div>I could use some advice on how best to test this code. Currently I have a little dummy application with the proper supervision tree to trigger this upgrade case. Here are the steps I use to reproduce:</div><div><br></div><div><a href="https://gist.github.com/da109fb6939ef7aac031">https://gist.github.com/da109fb6939ef7aac031</a></div>
<div><br></div><div>Note that this is using my topic branch so you can see the additional logging.</div><div><br></div><div>I noticed that there is a release_handler_SUITE_data directory in the sasl tests, should I just add another directory specifically for this and store the dummy application there for testing?</div><div><br></div><div>Thanks.</div><div><br></div><div>-Joe</div><div><br></div><div><br>-- <br><div style="font-size: medium; ">Name: Joseph A. Williams</div><div style="font-size: medium; ">Email: joe@joetify.com</div><div style="font-size: medium; ">Blog: http://www.joeandmotorboat.com/</div><div style="font-size: medium; ">Twitter: http://twitter.com/williamsjoe</div><br></div>
<p style="color: #A0A0A8;">On Tuesday, August 30, 2011 at 1:44 AM, Siri Hansen wrote:</p>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span><div><div>Great Joe - this is much better :)<div>Could you please add some tests for this also?</div><div>Thanks</div><div>/siri</div><div><br><div>2011/8/29 Joe Williams <span dir="ltr"><<a href="mailto:joe@joetify.com">joe@joetify.com</a>></span><br><blockquote type="cite"><div>
<div>Siri,</div><div><br></div><div>
Please fetch this branch again.
</div>
<div><br></div><div>I have added errors where I had my functions returning empty lists. I believe this should bubble up to release_handler causing a restart similar to the timeout behavior we currently have.</div>
<div><div><br></div><div>-Joe</div><div><br></div><div><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div>
<div style="font-size:medium">Blog: <a href="http://www.joeandmotorboat.com/" target="_blank">http://www.joeandmotorboat.com/</a></div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></div>
</div><div><div><p style="color:#A0A0A8">On Monday, August 29, 2011 at 9:35 AM, Joe Williams wrote:</p><blockquote type="cite"><div>
<span><div><div>
<div>Siri,</div><div><br></div><div>In case #2 the node would be in an "unpacked" state but perhaps that isn't possible since the upgrade may be partially installed already. I'll work on implementing #1 and reply back soon.</div>
<div><br></div><div>-Joe</div><div><br></div>
<div><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/</div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></div>
<p style="color:#A0A0A8">On Monday, August 29, 2011 at 7:31 AM, Siri Hansen wrote:</p><blockquote type="cite"><div>
<span><div><div>Hi Joe - <div>I think I would prefer solution 1), although that's probably mostly because I don't really understand solution 2)... What do you mean by "stop the upgrade from completing"? in which state would the node be after this?</div>
<div>/siri<br><br><div>2011/8/26 Joe Williams <span dir="ltr"><<a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a>></span><br><blockquote type="cite"><div>
<div>
Siri,
</div><div><br></div><div>That sounds correct, with the current patch there is that risk. In my case I would see the error message post-upgrade and restart things as needed but I certainly see your point. The VM restarting is a brutal but idiomatic way to deal with this issue, let it fail :).</div>
<div><br></div><div>I think there are two possibilities here, 1) continue with the restart behavior but make sure we print error messages before we do or 2) print error messages but stop the upgrade from completing if we catch the bad case. Thoughts?</div>
<div><div><br></div><div>-Joe</div><div><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div>
<div style="font-size:medium">Blog: <a href="http://www.joeandmotorboat.com/" target="_blank">http://www.joeandmotorboat.com/</a></div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></div>
</div><div><div><p style="color:#A0A0A8">On Friday, August 26, 2011 at 1:08 AM, Siri Hansen wrote:</p><blockquote type="cite"><div>
<span><div><div>Hi again, Joe!<div><br></div><div>Sorry for being so slow - but I still don't really understand :(</div><div>My concerns are really about whether or not we should allow the upgrade to be performed in this case. For sure I think we should </div>
<div><br></div><div>1) avoid the timeout, and</div><div>2) let the user know what the problem is</div><div><br></div><div>but is it correct to let the upgrade pass after this? Is it not an error situation?</div><div><br>
</div>
<div>It seems to me that we risk getting into a situation where we believe that the system is upgraded, but in fact there could be branches of the supervisor tree where process have not had the chance to run their code_change functions. I mean - even if we print the error report, there is no guarantee that it is really detected unless the operation actually fails.</div>
<div><br></div><div>Please correct me if I completely misunderstood the situation.</div><div><br></div><div>Regards</div><div>/siri</div><div><br></div><div><br><div>2011/8/25 Joe Williams <span dir="ltr"><<a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a>></span><br><blockquote type="cite"><div>
<div>
Siri,
</div><div><br></div><div>I ran into two issues that this patch addresses. Check out the commit message at <a href="https://github.com/joewilliams/otp/commit/9c3a53789326cdd929f1c3b4525716b1c0abfe87" target="_blank">https://github.com/joewilliams/otp/commit/9c3a53789326cdd929f1c3b4525716b1c0abfe87</a> for the details. In both cases I found that in production an error in the logs was preferable to the restart of the VM since both are easily fixable with a small application change or in the case of the suspended supervisor using a different app up. Also see this comment in release_handler_1 regarding the supervisor, <a href="https://github.com/erlang/otp/blob/dev/lib/sasl/src/release_handler_1.erl#L454" target="_blank">https://github.com/erlang/otp/blob/dev/lib/sasl/src/release_handler_1.erl#L454</a> which suggests this corner case is known by at least a few people. Currently there is no way to know *why* your VM just restarted after the upgrade in either case.</div>
<div><br></div><div>Let me know if you have any other questions.</div><div><div><br></div><div>-Joe</div><div><br></div><div><br></div><div>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">
Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">Blog: <a href="http://www.joeandmotorboat.com/" target="_blank">http://www.joeandmotorboat.com/</a></div><div style="font-size:medium">
Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div><br></div>
</div><div><div><p style="color:#A0A0A8">On Thursday, August 25, 2011 at 6:35 AM, Siri Hansen wrote:</p><blockquote type="cite"><div>
<span><div><div>Hi again, Joe!<div><br></div><div>Could you please explain a bit about the situation where you discovered this problem? I agree that the timeout and VM restart is not very good, and it makes sense to check if the supervisor is suspended. But I'm not really sure if it is correct to allow the upgrade to continue when this error occurs. Even if an error message is printed, I guess it could be quite easy to miss this fact... and the question is if that would be a problem or not? Why is the supervisor suspended in the first place?</div>
<div><br></div><div>Regards</div><div>/siri</div><div> <br><br><div>2011/8/25 Siri Hansen <span dir="ltr"><<a href="mailto:erlangsiri@gmail.com" target="_blank">erlangsiri@gmail.com</a>></span><br><blockquote type="cite"><div>
<div>
<div>
<div>
Hi Joe - I've just started looking at this. Do you think it would be possible to add a test case for it?<div>Regards</div><div>/siri<div><div><br><br><div>2011/8/24 Joe Williams <span dir="ltr"><<a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a>></span><br><blockquote type="cite"><div>
<div>
Anything I can do regarding this patch? I have happily been running it in production since I submitted it to the list in June.
</div><div><div><br></div><div>-Joe</div><div><br></div>
<div><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">
Blog: <a href="http://www.joeandmotorboat.com/" target="_blank">http://www.joeandmotorboat.com/</a></div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></div>
</div><div><div><p style="color:#A0A0A8">On Wednesday, July 6, 2011 at 3:43 PM, Joe Williams wrote:</p><blockquote type="cite"><div>
<span><div><div><div><div>
<div>
<span>Anything I can do to help this patch graduate?</span></div><div><span><br></span></div><div><span>Thanks!</span></div><div><span><br></span></div><div><span>-Joe</span></div><div><span><br>
</span>
<span><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/</div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></span>
<p style="color:#a0a0a0">On Tuesday, June 14, 2011 at 12:26 PM, Joe Williams wrote:</p><blockquote type="cite"><div>
<span><div><div><div>
<div>
<span>Updated this branch, please refetch.</span></div><div><br></div><div><span></span>git fetch git://<a href="http://github.com/joewilliams/otp.git" style="color:rgb(0, 58, 138)" target="_blank">github.com/joewilliams/otp.git</a> release_handler_1</div>
<div><br></div><div>
<span><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/</div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></span>
<p style="color:#a0a0a0">On Friday, June 10, 2011 at 8:52 AM, Joe Williams wrote:</p><blockquote type="cite"><div>
<span><div><div><div>
<div>
<span>Great, thanks!</span></div><div><span><br>
</span>
<span><br>-- <br><div style="font-size:medium">Name: Joseph A. Williams</div><div style="font-size:medium">Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a></div><div style="font-size:medium">
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/</div><div style="font-size:medium">Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a></div>
<br></span>
<p style="color:#a0a0a0">On Friday, June 10, 2011 at 8:51 AM, Raimo Niskanen wrote:</p><blockquote type="cite"><div>
<span><div><div>On Thu, Jun 09, 2011 at 08:20:51AM -0700, Joe Williams wrote:<br><blockquote type="cite"><div>Please fetch:<br><br>git fetch git://<a href="http://github.com/joewilliams/otp.git" target="_blank">github.com/joewilliams/otp.git</a> release_handler_1<br>
<br>This is a different branch with a better commit message and no white space changes.<br></div></blockquote><br>Excellent. I will include your patch in 'pu' after rewording the<br>summary line to imperative form.<br>
<br><blockquote type="cite"><div><br><br><br>-- <br>Name: Joseph A. Williams<br>Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a><br>Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/<br>
Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a><br><br><br>On Thursday, June 9, 2011 at 7:44 AM, Joe Williams wrote:<br><br><blockquote type="cite"><div>Nothing specific, just wondered if anyone had any thoughts on how I dealt with a couple of corner cases in installing releases.<br>
<br>I'll fix things up and get back shortly.<br><br>-- <br>Name: Joseph A. Williams<br>Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a> (<a href="mailto:joe@joetify.com" target="_blank">mailto:joe@joetify.com</a>)<br>
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/<br>Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a><br><br><br>On Thursday, June 9, 2011 at 12:11 AM, Raimo Niskanen wrote:<br>
<br><blockquote type="cite"><div><div>On Wed, Jun 08, 2011 at 03:41:37PM -0700, Joe Williams wrote:<br><blockquote type="cite"><div>Any thoughts/feedback on this patch? I realize it doesn't follow the guidelines (<a href="https://github.com/erlang/otp/wiki/Submitting-patches" target="_blank">https://github.com/erlang/otp/wiki/Submitting-patches</a>) exactly and will clean it up soon.<br>
</div></blockquote><br>Anything in particular? I just got caught up in tideous merge work<br>yesterday and missed to include your patch in 'pu', I was about<br>to take it now.<br><br>But if you have a cleanup I can wait for it...<br>
<br><blockquote type="cite"><div><br><br>-- <br>Name: Joseph A. Williams<br>Email: <a href="mailto:joe@joetify.com" target="_blank">joe@joetify.com</a> (<a href="mailto:joe@joetify.com" target="_blank">mailto:joe@joetify.com</a>)<br>
Blog: <a href="http://www.joeandmotorboat.com" target="_blank">http://www.joeandmotorboat.com</a>/<br>Twitter: <a href="http://twitter.com/williamsjoe" target="_blank">http://twitter.com/williamsjoe</a><br><br><br>On Tuesday, June 7, 2011 at 2:33 PM, Joe Williams wrote:<br>
<br><blockquote type="cite"><div> git fetch git://<a href="http://github.com/joewilliams/otp.git" target="_blank">github.com/joewilliams/otp.git</a> (<a href="http://github.com/joewilliams/otp.git" target="_blank">http://github.com/joewilliams/otp.git</a>) (<a href="http://github.com/joewilliams/otp.git" target="_blank">http://github.com/joewilliams/otp.git</a>) release_handler<br>
</div></blockquote></div></blockquote><br><blockquote type="cite"><div>_______________________________________________<br>erlang-patches mailing list<br><a href="mailto:erlang-patches@erlang.org" target="_blank">erlang-patches@erlang.org</a> (<a href="mailto:erlang-patches@erlang.org" target="_blank">mailto:erlang-patches@erlang.org</a>)<br>
<a href="http://erlang.org/mailman/listinfo/erlang-patches" target="_blank">http://erlang.org/mailman/listinfo/erlang-patches</a><br></div></blockquote><br><br>-- <br><br>/ Raimo Niskanen, Erlang/OTP, Ericsson AB<br></div>
</div></blockquote><br>_______________________________________________<br>erlang-patches mailing list<br><a href="mailto:erlang-patches@erlang.org" target="_blank">erlang-patches@erlang.org</a> (<a href="mailto:erlang-patches@erlang.org" target="_blank">mailto:erlang-patches@erlang.org</a>)<br>
<a href="http://erlang.org/mailman/listinfo/erlang-patches" target="_blank">http://erlang.org/mailman/listinfo/erlang-patches</a><br></div></blockquote></div></blockquote><br>-- <br><br>/ Raimo Niskanen, Erlang/OTP, Ericsson AB<br>
</div></div></span>
</div></blockquote><div>
<br>
</div>
</div>
</div></div></div></span>
</div></blockquote><div>
<br>
</div>
</div>
</div></div></div></span>
</div></blockquote><div>
<br>
</div>
</div>
</div></div></div></div></span>
</div></blockquote><div>
<br>
</div>
</div></div><br>_______________________________________________<br>
erlang-patches mailing list<br>
<a href="mailto:erlang-patches@erlang.org" target="_blank">erlang-patches@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-patches" target="_blank">http://erlang.org/mailman/listinfo/erlang-patches</a><br>
<br></div></blockquote></div><br></div></div></div>
</div></div></div></div></blockquote></div><br></div>
</div></div></span>
</div></blockquote><div>
<br>
</div>
</div></div></div></blockquote></div><br></div>
</div></div></span>
</div></blockquote><div>
<br>
</div>
</div></div></div></blockquote></div><br></div>
</div></div></span>
</div></blockquote><div>
<br>
</div>
</div><div>_______________________________________________<br>erlang-patches mailing list<br><a href="mailto:erlang-patches@erlang.org" target="_blank">erlang-patches@erlang.org</a><br><a href="http://erlang.org/mailman/listinfo/erlang-patches" target="_blank">http://erlang.org/mailman/listinfo/erlang-patches</a><br>
</div></div></span>
</div></blockquote><div>
<br>
</div>
</div></div></div></blockquote></div><br></div>
</div></div></span>
</blockquote>
<div>
<br>
</div>