[erlang-questions] Supervisor with better restart strategies?

[Essien Essien]

Hi Anthony,

On Wed, Aug 18, 2010 at 10:40 PM, Anthony Molinaro
<anthonym@REDACTED> wrote:
> Hi,
>
>  I've got a system which connects to another system.  I'd like a pool
> of connections to that other system.  I have something which works using
> a supervisor and pg2, however, the only restart strategy available in
> the supervisor doesn't play well with the back end service going down.
> I'd like some sort of supervisor which allows you to specify the number
> of attempted restarts with some backoff between tries (in other words
> instead of specifying {one_for_one, 10, 10} (which basically happens
> instantaneously when the backend is offline) I'd prefer something like
> {one_for_one, 10, 10, 100} (which would mean restart 10 times with 10
> second sleep up to 100 seconds total, or something like that).
> Is there something like this already out there somewhere?

I've been working on an SMPP gateway implementation and ran into this
same problem when for connecting with the SMSC over a link that could
go down.

What I did was first to implement a generic "backoff" module. The
algorithm is really really basic. It exposes:
                    backoff:register/4 - register a process with the
backoff module giving it a SPEC (pid, min, max, delta, MFA).
Internally, timer:apply is used to call the MFA after 'min' time has
passed. If the caller calls, increment/0, the wait time is updated to
min+delta and this continues untill it gets to 'max' and basically
stops there, untill the caller once again calls regular/0 then it
starts again from min.

                    backoff:deregister/0 - delete the previously added
spec and stop calling MFA
                    backoff:regular/0 - resets the wait counter
                    backoff:increment/0 - increments the internal wait
time by delta

With this generic backoff module, I now created a generic "nanny"
module. The nanny module does three things:

     start children - when starting, the nanny will attempt to start
all children using a specified start MFA. After this, it registers
with the backoff module sending it the babysit MFA.
     babysit children - when this is fired, the nanny walks through
its children list and starts up the non-alive children
    wake children - the nanny can be asked by an external process to
'wake' its children. I use this to prompt some of the children that
fetch tasks from a queue to proceed.

I've not put much thought in making this useful any more generally
than how I currently use these. This can most probably be done cleaner
and better, or a mechanism could be built as a patch to supervisor
according to your original suggestion (I initially) toyed with this
idea, but eventually went for this disconnected approach.

The entire project which makes use of this is not yet publicly
released, but will be soon, so I have attached specific files.
Hopefully, these shed some light on how they're used.

I hope this comes in helpful, even slightly :)

cheers,
Essien
>
> Thanks,
>
> -Anthony
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro                           <anthonym@REDACTED>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: backoff.erl
Type: text/x-erlang
Size: 4667 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20100819/cf4e49c1/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nanny.erl
Type: text/x-erlang
Size: 6089 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20100819/cf4e49c1/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rx_sup.erl
Type: text/x-erlang
Size: 1401 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20100819/cf4e49c1/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tx_sup.erl
Type: text/x-erlang
Size: 1401 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20100819/cf4e49c1/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mmayen.erl
Type: text/x-erlang
Size: 2468 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20100819/cf4e49c1/attachment-0004.bin>