From maria-12648430@REDACTED Thu Mar 4 13:47:14 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Thu, 4 Mar 2021 13:47:14 +0100 (CET) Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs Message-ID: <1555139477.285647.1614862034849@email.ionos.de> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: eep-supervisor-significant.md Type: application/octet-stream Size: 7422 bytes Desc: not available URL: From raimo+eeps@REDACTED Thu Mar 4 16:26:25 2021 From: raimo+eeps@REDACTED (Raimo Niskanen) Date: Thu, 4 Mar 2021 16:26:25 +0100 Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: <1555139477.285647.1614862034849@email.ionos.de> References: <1555139477.285647.1614862034849@email.ionos.de> Message-ID: <20210304152625.GA50498@erix.ericsson.se> It has been accepted as EEP 56. / RaimoNiskanen On Thu, Mar 04, 2021 at 01:47:14PM +0100, Maria Scott wrote: > > > > > > >
> Hello, >
>
>
>
>
> in the course of discussion around PR #4521 in OTP, it was decided by the OTP Technical Board that the PR will need an EEP. So here it is. We hope we did everything right, we tried hard to be clear, precise and exhaustive, and adhere to the requirements outlined in EEPs 1 and 33. But this is our first EEP ever, so... ^^; >
>
>
>
>
> Kind regards, >
>
>     Maria Scott >
>
>     Jan Uhlig >
> > > _______________________________________________ > eeps mailing list > eeps@REDACTED > http://erlang.org/mailman/listinfo/eeps -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jose.valim@REDACTED Thu Mar 4 16:45:49 2021 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Thu, 4 Mar 2021 16:45:49 +0100 Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: <1555139477.285647.1614862034849@email.ionos.de> References: <1555139477.285647.1614862034849@email.ionos.de> Message-ID: Thanks for sharing. I have read the proposal and I think it describes the problem well! I have only one minor comment on the proposed solution, which is this: > The new supervisor flag is named shutdown with possible values normal, any_significantand all_significant, with normal being the default. I don't like "normal" being the default because now I have to remember to change two places, the supervisor specification and the child spec, when configuring a significant child. The argument for this choice was: > With the supervisor shutdown flag set to normal, the child spec flag significant is ignored, even if present and set to true. This is intended as a safety means to defend against unwanted breaking of old code. I don't think it is possible for old code to break because there is no old code using significant in a child spec. :) Therefore I would propose for the default to be either any_significant or all_significant (if we want to be conversative, the latter). If we really think a default of normal is necessary, then I would propose to at least warn if the supervisor is normal and a significant child is given, as that will eventually save someone from debugging why the significant flag is not working as expected. :) I also think #{shutdown => normal} in a supervisor spec can be confusing, because someone may think it is customizing the exit reason of the supervisor, which is typically shutdown (and not normal). If normal is no longer the default, you could remove the normal option altogether, but if you want to keep it, perhaps something like ignore_significant is clearer? Thank you! On Thu, Mar 4, 2021 at 3:52 PM Maria Scott wrote: > Hello, > > in the course of discussion around PR #4521 > in OTP, it was decided by the > OTP Technical Board that the PR will need an EEP. So here it is. We hope we > did everything right, we tried hard to be clear, precise and exhaustive, > and adhere to the requirements outlined in EEPs 1 and 33. But this is our > first EEP ever, so... ^^; > > Kind regards, > Maria Scott > Jan Uhlig > _______________________________________________ > eeps mailing list > eeps@REDACTED > http://erlang.org/mailman/listinfo/eeps > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Mar 4 20:36:48 2021 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 4 Mar 2021 11:36:48 -0800 Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: References: <1555139477.285647.1614862034849@email.ionos.de> Message-ID: On 3/4/21 7:45 AM, Jos? Valim wrote: > Thanks for sharing. I have read the proposal and I think it describes > the problem well! > > I have only one minor comment on the proposed solution, which is this: > > > The new supervisor flag is named |shutdown| with possible values > |normal|, |any_significant|and |all_significant|, with |normal| being > the default. > > I don't like "normal" being the default because now I have to remember > to change two places, the supervisor specification and the child spec, > when configuring a significant child. The argument for this choice was: > > > With the supervisor |shutdown| flag set to |normal|, the child spec > flag |significant| is ignored, even if present and set to |true|. This > is intended as a safety means to defend against unwanted breaking of > old code. > > I don't think it is possible for old code to break because there is no > old code using significant in a child spec. :) > > Therefore I would propose for the default to be either any_significant > or all_significant (if we want to be conversative, the latter). If we > really think a default of normal is necessary, then I would propose to > at least warn if the supervisor is normal and a significant child is > given, as that will eventually save someone from debugging why the > significant flag is not working as expected. :) > > I also think #{shutdown => normal} in a supervisor spec can be > confusing, because someone may think it is customizing the exit reason > of the supervisor, which is typically shutdown (and not normal). If > normal is no longer the default, you could remove the normal option > altogether, but if you want to keep it, perhaps something like > ignore_significant is clearer? I would prefer the normal option remain, but with a different name. I agree using the name "normal" would be confusing.? However, the option should exist to mark a supervisor as part of a static supervision hierarchy.? There should also be error checking to ensure child specs with significant set to true cause an error (the value doesn't get silently ignored).? The error can be used as a return value for supervisor:start_child/2 and would block significant use where it is considered inappropriate (some supervisor processes would want to always exist as part of a static hierarchy). I am not sure about a name instead of "normal" for the option. Alternative name ideas for the option are "default", "none", "static", "external". Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Mar 5 14:26:06 2021 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 5 Mar 2021 08:26:06 -0500 Subject: [eeps] EEP 049: Value-Based Error Handling Mechanisms In-Reply-To: References: <20180904210702.GL801@ferdmbp.local> <20181002161629.GA5251@erix.ericsson.se> <20181003022701.GB38078@ferdmbp.local> <20181003070956.GA82674@erix.ericsson.se> <20181127084415.GA36141@erix.ericsson.se> Message-ID: On Fri, Dec 4, 2020 at 9:15 PM Fred Hebert wrote: > Before going for a reference implementation (I'd love to get this done in > time for OTP-24 but frankly I don't think I'll be solid enough to get it > ready), I decided to take the time to at least update the EEP as per the > comments in this thread and our latest discussions: > > https://github.com/erlang/eep/pull/18 > > This should now cover adjusted rationales, move all the older forms of the > proposal to an alternative version that was dismissed, and all the examples > have been rewritten to the new form to account for what was debated here. > > Hi, this hasn't moved since December, I was wondering if the new form of EEP was worth merging/discussing at this point before implementation work goes forth? -------------- next part -------------- An HTML attachment was scrubbed... URL: From maria-12648430@REDACTED Fri Mar 5 14:57:49 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Fri, 5 Mar 2021 14:57:49 +0100 (CET) Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: References: <1555139477.285647.1614862034849@email.ionos.de> Message-ID: <1607709471.290886.1614952669593@email.ionos.de> Hi Jos?, > > The new supervisor flag is named shutdown with possible values normal, any_significantand all_significant, with normal being the default. > > I don't like "normal" being the default because now I have to remember to change two places, the supervisor specification and the child spec, when configuring a significant child. The argument for this choice was: > > > With the supervisor shutdown flag set to normal, the child spec flag significant is ignored, even if present and set to true. This is intended as a safety means to defend against unwanted breaking of old code. > > I don't think it is possible for old code to break because there is no old code using significant in a child spec. :) Ok, granted. > Therefore I would propose for the default to be either any_significant or all_significant (if we want to be conversative, the latter). I'm not sure that either of the two is a good choice for a default value. Whichever you chose, it could happen that a child started later via `start_child` takes down the supervisor when it was declared as significant (probably by accident) and exits, which in turn would be something hard to debug by just looking at the supervisor code. A "dont't care about significant child exits" option allows the designer of a supervisor to defend against this, or rather, he has to explicitly allow it. (this reasoning is not contained in the EEP yet, I'll add it in the next revision) > If we really think a default of normal is necessary, then I would propose to at least warn if the supervisor is normal and a significant child is given, as that will eventually save someone from debugging why the significant flag is not working as expected. :) That sounds reasonable, yes. > I also think #{shutdown => normal} in a supervisor spec can be confusing, because someone may think it is customizing the exit reason of the supervisor, which is typically shutdown (and not normal). Yes, "normal" is probably not the best choice. We were thinking of using "never" before, but it may lead people to think that a supervisor with this setting will really never shut down. > If normal is no longer the default, you could remove the normal option altogether, but if you want to keep it, perhaps something like ignore_significant is clearer? I think this would be too specific, ie we were keeping this non-commital on purpose. It could be that more options on how to effect an automated shutdown are added in the future (though I can't think of any off the top of my head ;)), and then we would need another ignore_ option, and probably a way to combine them. Right now, I think we could change the name of the sup flag to something more specific like "auto_shutdown", and rename "normal" (which would not make sense any more then) to "never". What do you think? Kind regards, Maria Scott From maria-12648430@REDACTED Fri Mar 5 15:07:58 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Fri, 5 Mar 2021 15:07:58 +0100 (CET) Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: References: <1555139477.285647.1614862034849@email.ionos.de> Message-ID: <1245641897.291325.1614953278141@email.ionos.de> Hi Michael, > I would prefer the normal option remain, but with a different name. I agree using the name "normal" would be confusing. However, the option should exist to mark a supervisor as part of a static supervision hierarchy. There should also be error checking to ensure child specs with significant set to true cause an error (the value doesn't get silently ignored). The error can be used as a return value for supervisor:start_child/2 and would block significant use where it is considered inappropriate (some supervisor processes would want to always exist as part of a static hierarchy). Yes, I fully agree :) > I am not sure about a name instead of "normal" for the option. Alternative name ideas for the option are "default", "none", "static", "external". Thanks for the suggestions, but I think they somewhat miss the point in one way or another ^^; As I was already saying in my reply to Jos? Valim, I think it might be better to rename the sup flag to something like "auto_shutdown", and rename "normal" to "never". This way, it would be clearer as to what the option refers (_automated_ shutdown, as opposed to shutdown of any sort), and "never" would be clearer in that it declares to _never_ do an _automated_ (self-) shutdown. Kind regards, Maria Scott From mjtruog@REDACTED Fri Mar 5 22:57:34 2021 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 5 Mar 2021 13:57:34 -0800 Subject: EEP proposal - Automatic supervisor shutdown triggered by termination of significant childs In-Reply-To: <1245641897.291325.1614953278141@email.ionos.de> References: <1555139477.285647.1614862034849@email.ionos.de> <1245641897.291325.1614953278141@email.ionos.de> Message-ID: On 3/5/21 6:07 AM, Maria Scott wrote: >> I am not sure about a name instead of "normal" for the option. Alternative name ideas for the option are "default", "none", "static", "external". > Thanks for the suggestions, but I think they somewhat miss the point in one way or another ^^; > > As I was already saying in my reply to Jos? Valim, I think it might be better to rename the sup flag to something like "auto_shutdown", and rename "normal" to "never". This way, it would be clearer as to what the option refers (_automated_ shutdown, as opposed to shutdown of any sort), and "never" would be clearer in that it declares to _never_ do an _automated_ (self-) shutdown. > The "auto_shutdown" = "never" approach is a good way to make it clear that the supervisor is part of a static hierarchy and would only stop due to MaxR or manual manipulation (e.g., a supervisor:stop function if it was added). Thanks, Michael From maria-12648430@REDACTED Tue Mar 9 11:48:52 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Tue, 9 Mar 2021 11:48:52 +0100 (CET) Subject: EEP 56 Update Message-ID: <1524050085.12378.1615286932392@email.ionos.de> Hello, EEP 56 has been updated/improved, according to feedback received so far. http://erlang.org/eeps/eep-0056.html Changes: * Improved grammar (childs --> children, except in one place that I forgot) * Renamed the "shutdown" supervisor flag to "auto_shutdown" and the value "normal" to "never" * Clarified the purpose of the "auto_shutdown => never" setting * Added a paragraph about significant children in connection with simple_one_for_one supervisors More feedback welcome :) A specific point on which I would like more feedback is how a combination of "auto_shutdown => never" and child specs with "significant => true" (be it in the return from init/1 or given to start_child/2) settings. In actual meaning, the two options don't contradict each other, ie you may mark a child as significant, the supervisor just doesn't care in respect to auto-shutdown. It may be that this is rarely what is wanted in practice, but it also has no indisputable negative effects. So we think it should be accepted (ie, a "auto_shutdown => never" supervisor should not refuse to start a child marked as significant), but it might be good to log a warning in such a case. What do you think? A similar question concerns the combination of "restart => permanent" and "significant => true" in a child spec. A permanent child will always be restarted, it doesn't matter if it is marked as significant or not. Similar to what was said above, in meaning the two options don't contradict each other, but in practice you might care. Kind regards, Maria Scott From mjtruog@REDACTED Wed Mar 10 03:17:28 2021 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 9 Mar 2021 18:17:28 -0800 Subject: EEP 56 Update In-Reply-To: <1524050085.12378.1615286932392@email.ionos.de> References: <1524050085.12378.1615286932392@email.ionos.de> Message-ID: On 3/9/21 2:48 AM, Maria Scott wrote: > A specific point on which I would like more feedback is how a combination of "auto_shutdown => never" and child specs with "significant => true" (be it in the return from init/1 or given to start_child/2) settings. > In actual meaning, the two options don't contradict each other, ie you may mark a child as significant, the supervisor just doesn't care in respect to auto-shutdown. > It may be that this is rarely what is wanted in practice, but it also has no indisputable negative effects. > So we think it should be accepted (ie, a "auto_shutdown => never" supervisor should not refuse to start a child marked as significant), but it might be good to log a warning in such a case. What do you think? > > A similar question concerns the combination of "restart => permanent" and "significant => true" in a child spec. A permanent child will always be restarted, it doesn't matter if it is marked as significant or not. > Similar to what was said above, in meaning the two options don't contradict each other, but in practice you might care. Hi Maria, The supervisor behaviour is very central in Erlang/OTP and all other behaviours build upon it with the usage of their separate modules. My view is that the supervisor needs validation to ensure its process state space is as minimal as possible.? Ensuring its state is minimal makes it easier to diagnose, test, trust and utilize. I find it hard to argue for not having validation of arguments that are invalid.? You could say "We can't possibly consider every dumb thing a developer may do, so let's keep our code as minimal as possible so everything looks neat and simple in an effort to avoid bugs.".? However, if you choose to not block invalid options, it adds complexity that makes the Erlang process more difficult to understand (i.e., more difficult to understand "Why is this option here? What was this developer thinking?"). So, I would urge you to check for "significant == true" when either "auto_shutdown == never" or "restart == permanent", to block invalid use and ensure the supervisor process doesn't contain invalid state (even if the state doesn't cause functionality problems now, it can always become a bug in the future). A separate example of how the supervisor would benefit from more validation is when shutdown > 1000 * (MaxT / MaxR) (i.e., if shutdown and MaxR are both a pos_integer()).? That situation could be called the "immortal child" because the shutdown value can allow the child process to infinitely restart if it always takes longer than 1000 * (MaxT / MaxR) milliseconds to terminate.? That situation contains a bit of ambiguity because the time the child takes to terminate can vary randomly, so it may be a transient problem. However, it is able to make the supervisor ineffective for the child process. The "immortal child" bug may not have been fixed to avoid breaking legacy source code, I am not sure (hopefully legacy code doesn't exist with that bug).? The "significant == true" checks described above are focused on keeping the supervisor process state minimal and relevant, even though the option could be silently ignored if validation wasn't present.? Having the validation present is beneficial both for the development of the supervisor module and developing with it because it ensures the design lacks ambiguity and edge-cases (both can lead to bugs, either in the supervisor module or in the usage of the supervisor module). Best Regards, Michael From maria-12648430@REDACTED Wed Mar 10 11:41:52 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Wed, 10 Mar 2021 11:41:52 +0100 (CET) Subject: EEP 56 - question Message-ID: <1475285639.59141.1615372912723@email.ionos.de> Hello, it just occurred to me that, while the changes discussed in the EEP introduce no backwards incompatibilities, they are also fully forward compatible (if that is a term =^^=), and that there lies a catch. Imagine an application that uses the significant children feature as described in the EEP, in order to automatically shut down certain parts of it's supervision tree. As the new supervisor and child spec flags are new keys in the respective maps, this application will happily compile and run even on OTP versions predating this EEP, ignoring the new keys. However, the automatic shutdowns will never happen because the required mechanism is not in place in older OTP versions, and the application will be "leaking" supervisor processes. Any idea how we could tackle this, anyone? A safe way could be to add a new significant-related option to an existing key, instead of introducing new ones, at least for one of them. This would break in option value checking when used in an OTP version where the supervisor is not expecting the new value. I don't see any real nice place to fit this in, though. Another option might be to just document it and let implementors decide if and how they might defend against it, maybe with an -if(?OTP_RELEASE =< ...) directive. But who ever reads docs? ^^;;; Kind regards, Maria Scott From maria-12648430@REDACTED Wed Mar 10 12:51:13 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Wed, 10 Mar 2021 12:51:13 +0100 (CET) Subject: EEP 56 Update In-Reply-To: References: <1524050085.12378.1615286932392@email.ionos.de> Message-ID: <938970327.62444.1615377073876@email.ionos.de> Hi Michael, > So, I would urge you to check for "significant == true" when either > "auto_shutdown == never" or "restart == permanent", to block invalid use > and ensure the supervisor process doesn't contain invalid state (even if > the state doesn't cause functionality problems now, it can always become > a bug in the future). Ok, you certainly have a point. It is also easier to later allow things that were forbidden before (if the need should arise) than it would be the other way round. Kind regards, Maria From hans.r.nilsson@REDACTED Mon Mar 15 16:03:15 2021 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Mon, 15 Mar 2021 15:03:15 +0000 Subject: Sv: EEP 56 Update In-Reply-To: <938970327.62444.1615377073876@email.ionos.de> References: <1524050085.12378.1615286932392@email.ionos.de> , <938970327.62444.1615377073876@email.ionos.de> Message-ID: Hi Maria, I think this EEP could help the OTP SSH and I like your apporach to vitalizing Supervisors. Do you have a prototype? We are a bit busy now before OTP-23.3 and OTP-24.0 but I have some time to test if you want. Regards Hans Nilsson, Ericsson/OTP ________________________________ Fr?n: eeps f?r Maria Scott Skickat: den 10 mars 2021 12:51 Till: Michael Truog ; eeps@REDACTED ?mne: Re: EEP 56 Update Hi Michael, > So, I would urge you to check for "significant == true" when either > "auto_shutdown == never" or "restart == permanent", to block invalid use > and ensure the supervisor process doesn't contain invalid state (even if > the state doesn't cause functionality problems now, it can always become > a bug in the future). Ok, you certainly have a point. It is also easier to later allow things that were forbidden before (if the need should arise) than it would be the other way round. Kind regards, Maria _______________________________________________ eeps mailing list eeps@REDACTED https://protect2.fireeye.com/v1/url?k=3231a17b-6daa9b62-3231e1e0-8682aaa22bc0-edcef129dce45bbe&q=1&e=a795b4f6-8038-4f61-942c-8b3e40706bd4&u=http%3A%2F%2Ferlang.org%2Fmailman%2Flistinfo%2Feeps -------------- next part -------------- An HTML attachment was scrubbed... URL: From maria-12648430@REDACTED Mon Mar 15 16:45:10 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Mon, 15 Mar 2021 16:45:10 +0100 (CET) Subject: Sv: EEP 56 Update In-Reply-To: References: <1524050085.12378.1615286932392@email.ionos.de> , <938970327.62444.1615377073876@email.ionos.de> Message-ID: <2144050037.208559.1615823110906@email.ionos.de> Hi Hans, > I think this EEP could help the OTP SSH and I like your apporach to vitalizing Supervisors. Do you have a prototype? We are a bit busy now before OTP-23.3 and OTP-24.0 but I have some time to test if you want. We are currently preparing a reference implementation. In fact, the prototype itself has just been pushed to https://github.com/juhlig/otp/tree/eep56, and you can use that one already if you like. As it is, it already has all the functionality described in the current state of the EEP built in (and nothing more), you should be able to try it out as far as that goes. It is still largely untested, though, we only toyed with it a bit ^^; We want to add a few simple tests and test stubs, to show what we think should be tested before we publish it in a WIP-PR to OTP, and update the EEP document at the same time with what arrived as feedback here. That will probably be tomorrow. Kind regards, Maria From maria-12648430@REDACTED Mon Mar 15 16:58:29 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Mon, 15 Mar 2021 16:58:29 +0100 (CET) Subject: Sv: EEP 56 Update In-Reply-To: <2144050037.208559.1615823110906@email.ionos.de> References: <1524050085.12378.1615286932392@email.ionos.de> , <938970327.62444.1615377073876@email.ionos.de> <2144050037.208559.1615823110906@email.ionos.de> Message-ID: <2023640626.208821.1615823909475@email.ionos.de> > It is still largely untested, though, we only toyed with it a bit ^^; Ah... and dialyzer is complaining I see ^^; That will be fixed tomorrow. Regards, Maria From maria-12648430@REDACTED Tue Mar 23 16:20:28 2021 From: maria-12648430@REDACTED (Maria Scott) Date: Tue, 23 Mar 2021 16:20:28 +0100 (CET) Subject: EEP 56 - Update Message-ID: <464177029.571903.1616512828589@email.ionos.de> EEP 56 has been updated and augmented: https://github.com/erlang/eep/pull/21 The reference implementation has been equipped with a handful of tests: https://github.com/erlang/otp/pull/4638 Kind regards, Maria Scott