[erlang-questions] msg mailbox and gen_svr shutdown

Bob Ippolito <>
Tue May 15 06:42:04 CEST 2012


Yes, I want the other process to immediately take over. Requiring a load
balancer change isn't an elegant solution, I don't understand why you're
trying to avoid adding this useful functionality.

On Monday, May 14, 2012, Loïc Hoguin wrote:

> You stop listening after you're done gracefully stopping your currently
> running processes.
>
> For HTTP that could be something like:
>
> Set acceptors to 0.
> Optionally set 'onrequest' hook to reply with a 503 (for keepalives).
> Gracefully stop your processes.
> Stop the listener.
>
> Or do you need another process to take over immediately? Because in that
> case you usually don't need to use the same listening port, you can just
> change your firewall/lb rules for the port redirection from 80->P1 to
> 80->P2.
>
> On 05/15/2012 03:34 AM, Bob Ippolito wrote:
>
>> I don't think that is sufficient, you will need to stop listening as
>> well so the next OS process can take over.
>>
>> On Monday, May 14, 2012, Loïc Hoguin wrote:
>>
>>    This will be possible later on by reducing the number of acceptors
>>    to 0. This should be added sometimes this summer after the acceptor
>>    split happens in Cowboy.
>>
>>    On 05/14/2012 07:26 PM, Bob Ippolito wrote:
>>
>>        I agree that graceful shutdown is very application-specific.
>>        However,
>>        cowboy doesn't currently facilitate any sort of graceful
>>        shutdown unless
>>        you read the source code and poke directly at the appropriate
>>        supervisors like I did. The application specific stuff is easily
>>        done on
>>        your own with a process registry or a timeout, we used a
>>        combination of
>>        gproc and a timeout if things didn't shut down in an acceptable
>>        time frame.
>>
>>        I would suggest something like cowboy:stop_listener/1, maybe
>>        something
>>        like cowboy:stop_listening/1 or cowboy:stop_accepting/1.
>>
>>        On Sun, May 13, 2012 at 11:02 PM, Loïc Hoguin <
>>        <mailto:>> wrote:
>>
>>            Right, this is pretty much what I said to Paul in PM. It was in
>>            R14B03 that the behavior changed. I apparently have a @todo
>>        wrong
>>            past that release, and will take a look if I find other
>>        things to
>>            fix in the docs.
>>
>>            Copy pasting my private reply on this:
>>
>>            Ultimately if we remove the listener I think we want to stop
>>        everything.
>>
>>            For "server is overloaded" situations, you can very well use
>> the
>>        'onrequest' hook which can be set or changed dynamically through
>>            cowboy:set_protocol_options, in addition to giving it in
>>            start_listener. Takes a fun that has a single arg as a Req,
>>        returns
>>            a Req, and if you replied from within it it doesn't dispatch
>> the
>>            request and stops there (you can also force close the
>>        connection by
>>            setting the Connection header to "close").
>>
>>            And adding this:
>>
>>            For graceful shutdowns, well it's highly application
>>        dependent. Some
>>            apps are just short lived connections, so not accepting
>> requests
>>            plus a short delay ought to do it. Some are long lived, which
>>            probably requires to send a shutdown message. Some apps may
>> have
>>            connections critical enough that you don't want to shutdown
>>        them.
>>            It's up to the application implementor to devise the
>>        strategy to use
>>            for stopping.
>>
>>
>>            On 05/14/2012 02:34 AM, Fred Hebert wrote:
>>
>>                This is just a guess, but is it possible that this is
>>        due to the
>>                fact
>>                Cowboy is using simple_one_for_one supervision? In Pre
>>        R15, if I
>>                recall
>>                correctly, the shutdown of sofo supervisors was
>>        asynchronous. The
>>                supervisor would just die and let its children figure
>>        out it was
>>                gone.
>>
>>                Starting with R15, things started being synchronous and the
>>                supervisor
>>                would wait. A brutal kill that made things work fine
>>        before R15
>>                (excluding the issue of the application master killing
>>                everything) could
>>                start breaking in later versions.
>>
>>                Again, this is just a guess, without looking at the
>>        source or
>>                anything.
>>
>>                On Sun May 13 14:12:18 2012, Bob Ippolito wrote:
>>
>>                    Yes, it is documented that request processes
>>        continue after
>>                    you stop
>>                    the listener. This is incorrect.
>>
>>                    On Sunday, May 13, 2012, Anthony Ramine wrote:
>>
>>                    Paweł, from what I gather your supervisor waits 1 ms
>>        before
>>                    killing its children; I think it's pretty normal
>>        that some
>>                    messages are left unprocessed.
>>                    Bob, cowboy_requests_sup' shutdown strategy is
>>        brutal_kill,
>>                    is it
>>                    documented somewhere that Cowboy waits before
>>        killing requests'
>>                    processes?
>>
>>                    Regards,
>>
>>                    --
>>                    Anthony Ramine
>>
>>
>>
>>
>>                    Le 10 mai 2012 à 22:37, Bob Ippolito a écrit :
>>        https://gist.github.com/____**2655724<https://gist.github.com/____2655724>
>>        <https://gist.github.com/__**2655724<https://gist.github.com/__2655724>
>> >
>>        <https://gist.github.com/__**2655724<https://gist.github.com/__2655724>
>>        <https://gist.github.com/**2655724<https://gist.github.com/2655724>
>> >>
>>
>>                        On Thu, May 10, 2012 at 2:53 AM, Paweł Peregud
>>        < <mailto:>
>>        <javascript:_e({}, 'cvml',
>>        '
>>        <mailto:>');>> wrote:
>>
>>                        I was having fun with supervisors yesterday
>>        (Cowboy seems to
>>                        fail to fulfill the promise of not killing
>>        request processes
>>                        after listener removal) and I have an example.
>>        I've only
>>                        investigated the case when supervisor is killed,
>>        so YMMV.
>>                        Example code is attached. You may modify it to
>>        check the
>>                        behavior in your case.
>>
>>                        Start supervisor tree with
>>        exp_sup_sup:start_link(). Execute
>>                        test with exp_sup_sup:test() and
>>        exp_sup_sup:test_simple().
>>
>>                        In case of dying supervisor the answer is "no,
>>        it does not".
>>
>>                        When supervisor dies, your process is killed as
>>        via link
>>                        mechanism, so it may leave some unprocessed
>>        messages in
>>                        inbox. To make sure that every delivered message
>>        is served,
>>                        you need to add process_flag(trap_exit, true).
>>        Messages that
>>                        are sent after the moment when supervisor dies
>>        are not
>>                        processed.
>>
>>                        Best regards,
>>
>>                        Paul.
>>
>>
>>                        On May 9, 2012 11:06 AM, "Andy Richards"
>>        <**____com
>>        <mailto:>
>>        <javascript:_e({}, 'cvml',
>>        '**____com
>>        <mailto:>');>> wrote:
>>
>>                        Hi,
>>
>>                        I can't seem to see any confirmation in the
>>        documentation
>>                        so was wondering if anyone could confirm if
>>        messages are
>>                        still sent to a supervised gen_svr following a
>>        shutdown
>>                        message?
>>
>>                        If so how do I cleanly shutdown my gen_svr without
>>                        loosing messages? I read in the supervisor child
>>        spec
>>                        that a shutdown can be set to infinity which i
>> hoped
>>                        would allow me to process the msg's in my
>>        mailbox but if
>>                        I do this will my module continue to receive
>>        messages
>>                        from other processes? Is my approach flawed and
>>        if so
>>                        what other ways are there to cleanly shutting
>>        down my
>>                        gen_svr without loosing messages?
>>
>>                        Many thanks,
>>
>>                        Andy.
>>                        ______________________________**
>> _____________________
>>                        erlang-questions mailing list
>>        
>>        <mailto:> <javascript:_e({},
>>        'cvml',
>>        '
>>        <mailto:>')__;>
>>        http://erlang.org/mailman/____**listinfo/erlang-questions<http://erlang.org/mailman/____listinfo/erlang-questions>
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>> >
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>>        <http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>> >>
>>
>>
>>                        ______________________________**
>> _____________________
>>                        erlang-questions mailing list
>>        
>>        <mailto:> <javascript:_e({},
>>        'cvml',
>>        '
>>        <mailto:>')__;>
>>        http://erlang.org/mailman/____**listinfo/erlang-questions<http://erlang.org/mailman/____listinfo/erlang-questions>
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>> >
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>>        <http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>> >>
>>
>>
>>                        ______________________________**
>> _____________________
>>                        erlang-questions mailing list
>>        
>>        <mailto:> <javascript:_e({},
>>        'cvml',
>>        '
>>        <mailto:>')__;>
>>        http://erlang.org/mailman/____**listinfo/erlang-questions<http://erlang.org/mailman/____listinfo/erlang-questions>
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>> >
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>>        <http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>> >>
>>
>>
>>
>>
>>                    ______________________________**_____________________
>>                    erlang-questions mailing list
>>         <mailto:>
>>        http://erlang.org/mailman/____**listinfo/erlang-questions<http://erlang.org/mailman/____listinfo/erlang-questions>
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>> >
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>>        <http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>> >>
>>
>>                ______________________________**_____________________
>>                erlang-questions mailing list
>>         <mailto:>
>>        http://erlang.org/mailman/____**listinfo/erlang-questions<http://erlang.org/mailman/____listinfo/erlang-questions>
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>> >
>>        <http://erlang.org/mailman/__**listinfo/erlang-questions<http://erlang.org/mailman/__listinfo/erlang-questions>
>>        <http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>> >>
>>
>>
>>
>>            --
>>            Loïc Hoguin
>>            Erlang Cowboy
>>            Nine Nines
>>
>>
>>
>>
>>    --
>>    Loïc Hoguin
>>    Erlang Cowboy
>>    Nine Nines
>>
>>
>
> --
> Loïc Hoguin
> Erlang Cowboy
> Nine Nines
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120514/19f7d545/attachment.html>


More information about the erlang-questions mailing list