[erlang-questions] msg mailbox and gen_svr shutdown

Mon May 14 19:26:11 CEST 2012

I agree that graceful shutdown is very application-specific. However,
cowboy doesn't currently facilitate any sort of graceful shutdown unless
you read the source code and poke directly at the appropriate supervisors
like I did. The application specific stuff is easily done on your own with
a process registry or a timeout, we used a combination of gproc and a
timeout if things didn't shut down in an acceptable time frame.

I would suggest something like cowboy:stop_listener/1, maybe something like
cowboy:stop_listening/1 or cowboy:stop_accepting/1.

On Sun, May 13, 2012 at 11:02 PM, Loïc Hoguin <essen@REDACTED> wrote:

> Right, this is pretty much what I said to Paul in PM. It was in R14B03
> that the behavior changed. I apparently have a @todo wrong past that
> release, and will take a look if I find other things to fix in the docs.
>
> Copy pasting my private reply on this:
>
> Ultimately if we remove the listener I think we want to stop everything.
>
> For "server is overloaded" situations, you can very well use the
> 'onrequest' hook which can be set or changed dynamically through
> cowboy:set_protocol_options, in addition to giving it in start_listener.
> Takes a fun that has a single arg as a Req, returns a Req, and if you
> replied from within it it doesn't dispatch the request and stops there (you
> can also force close the connection by setting the Connection header to
> "close").
>
> And adding this:
>
> For graceful shutdowns, well it's highly application dependent. Some apps
> are just short lived connections, so not accepting requests plus a short
> delay ought to do it. Some are long lived, which probably requires to send
> a shutdown message. Some apps may have connections critical enough that you
> don't want to shutdown them. It's up to the application implementor to
> devise the strategy to use for stopping.
>
>
> On 05/14/2012 02:34 AM, Fred Hebert wrote:
>
>> This is just a guess, but is it possible that this is due to the fact
>> Cowboy is using simple_one_for_one supervision? In Pre R15, if I recall
>> correctly, the shutdown of sofo supervisors was asynchronous. The
>> supervisor would just die and let its children figure out it was gone.
>>
>> Starting with R15, things started being synchronous and the supervisor
>> would wait. A brutal kill that made things work fine before R15
>> (excluding the issue of the application master killing everything) could
>> start breaking in later versions.
>>
>> Again, this is just a guess, without looking at the source or anything.
>>
>> On Sun May 13 14:12:18 2012, Bob Ippolito wrote:
>>
>>> Yes, it is documented that request processes continue after you stop
>>> the listener. This is incorrect.
>>>
>>> On Sunday, May 13, 2012, Anthony Ramine wrote:
>>>
>>> Paweł, from what I gather your supervisor waits 1 ms before
>>> killing its children; I think it's pretty normal that some
>>> messages are left unprocessed.
>>> Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it
>>> documented somewhere that Cowboy waits before killing requests'
>>> processes?
>>>
>>> Regards,
>>>
>>> --
>>> Anthony Ramine
>>>
>>>
>>>
>>>
>>> Le 10 mai 2012 à 22:37, Bob Ippolito a écrit :
>>>
>>>  I had the same problem with Cowboy recently, really ought to file
>>>> an issue to at least fix the docs for that.
>>>>
>>>> For my app I wanted to implement a graceful shutdown that stopped
>>>> accepting new connections but allowed existing requests to
>>>> finish. The waiting part I implemented by adding a gproc local
>>>> property to the workers that I wanted to wait for and then
>>>> monitoring them.
>>>>
>>>> Looks like this:
>>>> https://gist.github.com/**2655724 <https://gist.github.com/2655724>
>>>>
>>>> On Thu, May 10, 2012 at 2:53 AM, Paweł Peregud
>>>> <paulperegud@REDACTED <javascript:_e({}, 'cvml',
>>>> 'paulperegud@REDACTED');>> wrote:
>>>>
>>>> I was having fun with supervisors yesterday (Cowboy seems to
>>>> fail to fulfill the promise of not killing request processes
>>>> after listener removal) and I have an example. I've only
>>>> investigated the case when supervisor is killed, so YMMV.
>>>> Example code is attached. You may modify it to check the
>>>> behavior in your case.
>>>>
>>>> Start supervisor tree with exp_sup_sup:start_link(). Execute
>>>> test with exp_sup_sup:test() and exp_sup_sup:test_simple().
>>>>
>>>> In case of dying supervisor the answer is "no, it does not".
>>>>
>>>> When supervisor dies, your process is killed as via link
>>>> mechanism, so it may leave some unprocessed messages in
>>>> inbox. To make sure that every delivered message is served,
>>>> you need to add process_flag(trap_exit, true). Messages that
>>>> are sent after the moment when supervisor dies are not processed.
>>>>
>>>> Best regards,
>>>>
>>>> Paul.
>>>>
>>>>
>>>> On May 9, 2012 11:06 AM, "Andy Richards"
>>>> <andy.richards.iit@REDACTED**com <andy.richards.iit@REDACTED><javascript:_e({}, 'cvml',
>>>> 'andy.richards.iit@REDACTED**com <andy.richards.iit@REDACTED>');>>
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I can't seem to see any confirmation in the documentation
>>>> so was wondering if anyone could confirm if messages are
>>>> still sent to a supervised gen_svr following a shutdown
>>>> message?
>>>>
>>>> If so how do I cleanly shutdown my gen_svr without
>>>> loosing messages? I read in the supervisor child spec
>>>> that a shutdown can be set to infinity which i hoped
>>>> would allow me to process the msg's in my mailbox but if
>>>> I do this will my module continue to receive messages
>>>> from other processes? Is my approach flawed and if so
>>>> what other ways are there to cleanly shutting down my
>>>> gen_svr without loosing messages?
>>>>
>>>> Many thanks,
>>>>
>>>> Andy.
>>>> ______________________________**_________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED <javascript:_e({}, 'cvml',
>>>> 'erlang-questions@REDACTED')**;>
>>>> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>>>>
>>>>
>>>> ______________________________**_________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED <javascript:_e({}, 'cvml',
>>>> 'erlang-questions@REDACTED')**;>
>>>> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>>>>
>>>>
>>>> ______________________________**_________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED <javascript:_e({}, 'cvml',
>>>> 'erlang-questions@REDACTED')**;>
>>>> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>>>>
>>>
>>>
>>>
>>> ______________________________**_________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>>>
>> ______________________________**_________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>>
>
>
> --
> Loïc Hoguin
> Erlang Cowboy
> Nine Nines
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120514/9372a397/attachment.htm>