[erlang-questions] gen_statem and multiple timeout vs 1

[Raimo Niskanen]

On Mon, Oct 03, 2016 at 02:30:32PM +0000, Vans S wrote:
> 
> 
> > > 
> > > If the implementation is quite simple using timers, I would not mind putting in the pull request.  I just came here to discuss about it and see if it is something that fits.
> > > 
> > > The rational is using erlang:send_after/3 is not enough in more complex cases, we need to also be stopping and starting new timers.  
> >
> > Does not erlang:start_timer/3,4 work in this case?  You can have any number
> > of such timers simultaneously running just as for erlang:send_after/3,4.
> 
> The example I gave was poor I forgot one key point and that was a case where cancel_timer was required. Say every scream resets your running.  So every 500ms a scream event would return
> [{timeout, 500,scream }, {timeout, 100, running_step}].
> 
> Using erlang:send_after we would have to cancel_timer on the running_step, also I am not sure how timers work, but if a function took long would a timer msg arrive in the mailbox? Thus cancel_timer would not work ideally? Say if the erlang:send_after is in 100ms, and we spend 200ms in a function then at the end of that function we call cancel_timer.

Using erlang:cancel_timer correctly needs at least this trick: after having
called erlang:cancel_timer you are guaranteed that no timeout message will
be delivered to you after that, so it is self to do a receive on the
TimerRef with 'after 0' to read out the timeout message.  Then you can check
the return value from erlang:cancel_timer/1,2 to decide when you for sure
do not need to read out the timeout message...

So it can function ideally if used wisely.

> 
> 
> > Here is an attempt:
> > 
> > You return [{named_timeout,Time,Name}] and will get an event
> > named_timeout,Name.  If you return [{named_timeout,NewTime,Name}] with the
> > Name of a running timer it will restart with NewTime.  Therefore you can
> > cancel it with [{named_timeout,infinity,Name}].  Any number of these can
> > run simultaneously and you distinguish them by Name.
> > 
> > Or is the benefit of this i.e just to hide the TimerRef behind the timer
> > Msg/Name too small to be worthy of implementing?
> 
> I think the benefit is two fold, first is not managing cancel timer, and second is the code would be cleaner to read and reason about.
> 
> 'infinity' as the timeout perhaps is misleading, could a new atom be introduced for this, perhaps cancel?  
> {timeout, cancel, scream}.

That would be possible and have its merits.

> 
> Crash if the scream timer is not initiated. If we cancel a timer that has to exist, we are in an undefined state.

Then 'infinity' can be used as: cancel if started otherwise ignore...

> 
> 'infinity' is fine for practicality purposes or the maximum value of a 56 bit integer (if that is the correct value before venturing into big Erlang integers).

Since the bignum limit is platform dependent the atom 'infinity' is
interpreted by all code at the appliction API level I know of handling timer
values to mean "no timer".  For example gen_server:call, et.al.  It is even
so that the pre-defined Dialyzer type timeout() is defined as
'infinity' | non_neg_integer() % integer() >= 0.

> 
> What is the benefit of {named_timeout,Time,Name} vs the current {timeout,Time,EventName}?

The current {timeout,Time,EventContent} will be cancelled by any other event
you receive.  It is intended to be used as some kind of inactivity timeout.
So if you get for example a call event for status that is handled in your
"handle in any state" code, it will cancel the timeout.

The suggested (and in the pipeline) {state_timeout,Time,EventContent}
will be cancelled by a state change and there can be only one such timer.
So typically events handled in "handle in any state" code will
not cancel this timer since unless changing states.

{named_timeout,Time,Name} would give you full control of when to start and
cancel any number of timers distinguished by Name, which is easier to read from
the code than handling TimerRefs.  It raises the abstraction level by
hiding the TimerRef and how to do a correct canceling at the price of
not being able to have multiple timers with the same Name, which you can do
with erlang:start_timer.

/ Raimo



> 
> 
>  
> 
>     On Monday, October 3, 2016 5:45 AM, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote:
>  
> 
>  On Fri, Sep 30, 2016 at 02:25:13PM +0000, Vans S wrote:
> > The reasoning behind this is because I am using a handle_event_function callback mode. I find this is more flexible.  A standard state machine can be in 1 state at 1 time.
> > 
> > But things are more complex now and the desire for a state machine to be in multiple states at 1 time is crucial.  
> > 
> > For example you can be running and screaming. Not only running then stopping to transition to screaming, then stop screaming and start running again.
> > Maybe this is beyond the scope of a state machine, excuse if my terminology is off.
> 
> I'd say that your state machine can only be in one state at any given time
> but you have a complex state as in multi dimensional and each combination
> of the different dimensions counts as a different state.  So you have 4
> discrete states: {not screaming, not running}, {not screaming, running},
> {screaming, not running} and {screaming, running} but wants to reason about
> the state as one complex term: {IsScreaming, IsRunning}.
> 
> > 
> > If the implementation is quite simple using timers, I would not mind putting in the pull request.  I just came here to discuss about it and see if it is something that fits.
> > 
> > The rational is using erlang:send_after/3 is not enough in more complex cases, we need to also be stopping and starting new timers.  
> 
> Does not erlang:start_timer/3,4 work in this case?  You can have any number
> of such timers simultaneously running just as for erlang:send_after/3,4.
> 
> > 
> > An example of this is say if we are in a running state and every 100 ms we timeout, when we timeout we take an extra step and update our position in the world, also we determine how fast we are running and maybe the next step will be in 80ms now.
> > 
> > Now while we are running, we also got a moral boost by seeing a well lit street, so in 5000ms our moral boost will expire, and will timeout.
> > 
> > Also we start screaming due to panic, every 500ms we send a scream to all nearby recipients.
> > 
> > Maybe this is all beyond the scope of a state machine.  But for me it seems the only change required to support this is allowing multiple timers, and Erlang always had the design philosophy of "do what works" vs "do what is academically sound".
> 
> The question is if it is possible to create a timer concept in gen_statem
> that is easier to use and still as flexible as erlang:start_timer/3,4 +
> erlang:cancel_timer/1,2.
> 
> With those primitives you have to handle the TimerRef yourself.
> 
> Here is an attempt:
> 
> You return [{named_timeout,Time,Name}] and will get an event
> named_timeout,Name.  If you return [{named_timeout,NewTime,Name}] with the
> Name of a running timer it will restart with NewTime.  Therefore you can
> cancel it with [{named_timeout,infinity,Name}].  Any number of these can
> run simultaneously and you distinguish them by Name.
> 
> Or is the benefit of this i.e just to hide the TimerRef behind the timer
> Msg/Name too small to be worthy of implementing?
> 
> / Raimo
> 
> 
> > 
> > 
> >  
> > 
> >    On Thursday, September 29, 2016 9:30 AM, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote:
> >  
> > 
> >  After giving this a second thought i wonder if a single state timer would
> > be a desired feature and enough in your case.
> > 
> > Today we have an event timeout, which is seldom useful since often you are
> > in one state and wait for something specific while stray events that either
> > are ignored or immediately replied to passes by.  The event timeout is
> > reset for every stray event.
> > 
> > What I think would cover many use cases is a single state timeout.  It
> > would be cancelled if you change states.  If you set it again the running
> > timer is cancelled and a new is started.  There would only need to be one
> > such timer so it is roughly as easy to implement as the event timeout.
> > 
> > There would be no way to cancel it other than changing states.
> > It would be started with an action {state_timeout,T,Msg}.
> > 
> > We should keep the old {timeout,T,Msg} since it is inherited from gen_fsm
> > and has some use cases.
> > 
> > What do you think?
> > 
> > / Raimo
> > 
> > 
> > 
> > On Mon, Sep 26, 2016 at 04:56:25PM +0200, Raimo Niskanen wrote:
> > > On Sun, Sep 25, 2016 at 05:32:19PM +0000, Vans S wrote:
> > > > Learning the new gen_statem made me desire for one extra feature.
> > > > 
> > > > Say you have a common use case of a webserver /w websockets, you have a general connection timeout of 120s, if no messages are received in 120s it means the socket is in an unknown state, and should be closed.
> > > > 
> > > > So you write your returns like this anytime the client sends you a message:
> > > > 
> > > > {next_state, NextState, NewData, {timeout, 120*1000, idle_timeout}}
> > > > 
> > > > Now if the client does not send any messages in 120 seconds, we will get a idle_timeout message sent to the gen_statem process.
> > > > 
> > > > Awesome.
> > > > 
> > > > But enter a little complexity, enter websockets.
> > > > 
> > > > Now we need to send a ping from the gen_statem every 15s to the client, but we also need to consider if we did not get any messages from the client in 120s, we are in unknown state and should terminate the connection.
> > > > 
> > > > So now we are just itching to do this on init:
> > > > 
> > > > {ok, initial_state, Data, [        {timeout, 120*1000, idle_timeout},        {timeout, 15*1000, websocket_ping}
> > > >     ]}
> > > > 
> > > > This way we do not need to manage our own timers using erlang:send_after.  timer module is not even a consideration due to how inefficient it is at scaling.
> > > > 
> > > > But of course we cannot do this, the latest timeout will always override any previous.
> > > > 
> > > > What do you think?
> > > 
> > > Your use case is in the middle ground between the existing event timeout
> > > and using erlang:start_timer/4,3, and is a special case of using
> > > erlang:start_timer/4,3.
> > > 
> > > The existing {timeout,T,Msg} is an *event* timeout, so you get either an
> > > event or the timeout.  The timer is cancelled by the first event.
> > > This semantics is easy to reason about and has got a fairly simple
> > > implementation in the state machine engine partly since it only needs
> > > to store one timer ref.
> > > 
> > > It seems you could use a state timeout, i.e the timeout is cancelled when
> > > the state changes.  This would require the state machine engine to hold any
> > > number of timer refs and cancel all during a state change.
> > > 
> > > This semantics is subtly similar to the current event timeout.  It would
> > > need a new option, e.g {state_timeout,T,Msg}.
> > > 
> > > The {state_timeout,_,_} semantics would be just a special case of using
> > > erlang:start_timer/4,3, keep your timer ref in the server state and cancel
> > > it when needed, since in the general case you might want to cancel the
> > > timer at some other state change or maybe not a state change but an event.
> > > 
> > > So the question is if a {state_timeout,_,_} feature that auto cancels the
> > > timer at the first state change is so useful that it is worthy of being
> > > implemented?  It is not _that_ much code that is needed to store
> > > a timer ref and cancel the timer started with erlang:start_timer/4,3,
> > > and it is more flexible.
> > > 
> > > I implemented the {timeout,_,_} feature just so make it easy to port from
> > > gen_fsm.  Otherwise I thought that using explicit timers was easy enough.
> > > 

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB