Re: Supervisor and configuration update — how to do it?

[Oleg Tarasenko]

Hey Max,

We also had a similar problem with our production. Our configuration (and
states) were results of computations over a long period of time.
The system itself was managed by Mesos, so it was hard to use erlang hot
updates, as it was suggested earlier (another topic, is
that hot updates are not trivial when you have multiple stateful processes).

Losing the state was quite painful for us, as it would require at least a
couple of minutes to restore it. Which would result
in at least 100K failed requests from the customers. And because of our
SLAs, it would have cost us quite a bit.

What worked for is Redis caching. It gave us reasonable performance + was
external to our system (so it was easier to
re-boot nodes).

Best regards,
Oleg



On Wed, May 20, 2020 at 11:23 PM Ameretat Reith <ameretat.reith@REDACTED>
wrote:

> I used to separate worker parameters to dynamic or static; static
> parameters
> would be set by supervisor but dynamic ones would be inducted from
> application
> env on worker initialization _or_ at runtime. So, each worker needing
> dynamic
> parameters, would have a `do_load_option` function invoking
> `application:get_env`  for each interesting dynamic parameter, and it
> would be
> called on worker initialization, e.g. in `init/1` for gen_servers then
> populate state. And like your approach, there was a `handle_call` to invoke
> `do_load_options` and reset state. This message would be sent on to workers
> found by `supervisor:which_children`.
>
> I had scripts (Juju hooks and then a homebuilt deployment system hook) that
> could set application env in sys.config on configuration changes. Then,
> that
> hook would call (rpcterms) a rpc on release to reload `sys.config`. This
> `reload_sys_config` calls `application_controller:change_application_data`
> to
> update envs and then procedures to call `do_load_options` on workers.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200521/32616447/attachment.htm>