[erlang-questions] gen_fsm - calling state function clauses directly (question from Learn You Some Erlang chapter 20)

[Raimo Niskanen]

On Fri, Aug 12, 2016 at 06:16:34PM +1000, Richard McLean wrote:
> Hi,
> 
> I’m just learning erlang/OTP so please excuse me if I ask a newbie question.
> 
> I’m currently reading "Learn You Some Erlang For Great Good” by Fred Hébert.
> 
> Regarding Chapter 20 “The Count of Applications”…
> 
> The author presents an OTP application that uses gen_fsm to recursively grep through a directory tree using one worker process per file/regex.
> 
> There are a couple of instances where the code transitions from one state to another (and simulates triggering an event) by calling a state function directly rather than via a {next_state, StateName, StateData} return tuple.
> 
> For example (page 326) :
> 
> dispatching(done, Data) ->
>     %% This is a special case. We cannot assume that all messages have NOT
>     %% been received by the time we hit 'done'. As such, we directly move to
>     %% listening/2 without waiting for an external event.
>     listening(done, Data).
> 
> listening(done, #data{regex=Re, refs=[]}) -> % all received!
>     [io:format("Regex ˜s has ˜p results˜n", [R,C]) || {R, C} <- Re],
>     {stop, normal, done};
> listening(done, Data) -> % entries still missing
>     {next_state, listening, Data}.
> 
> And (page 328) :
> 
> handle_event({complete, Regex, Ref, Count}, State,
>  Data = #data{regex=Re, refs=Refs}) ->
>     {Regex, OldCount} = lists:keyfind(Regex, 1, Re),
>     NewRe = lists:keyreplace(Regex, 1, Re, {Regex, OldCount+Count}),
>     NewData = Data#data{regex=NewRe, refs=Refs--[Ref]},
>     case State of
>         dispatching ->
>             {next_state, dispatching, NewData};
>         listening ->
>             listening(done, NewData)
>     end.
> 
> *dispatching and listening are the two states implemented in this gen_fsm with state functions of matching name.
> 
> I’m specifically referring to the  "listening(done, Data)” call that is the last (tail) call of the dispatching(done, Data) clause, and the listening(done, NewData) call that is one of the possible last (tail) calls of the handle_event({complete, Regex, Ref, Count}, State, Data = #data{regex=Re, refs=Refs}) clause.
> 
> I understand that the author is trying to simultaneously change state and execute the correct clause without having to send any events, and that just returning a {next_state, listening, StateData} tuple would only change the state… and to execute a clause in the listening state would then require matching a particular event (ie. “done”), which would mean having to send a “done” event, and other events or messages in the queue may need to be processed in the meantime which could leave the FSM in a state other than “listening” (ie. "dispatching”) which might lead to some oscillating behaviour.
> 
> My main question is…
> 
> Is this approach recommended or not ?  
> (and if not are there any other recommended ways to achieve the same thing ?)

There is no other way to do it in gen_fsm, except possibly for to
reorganize your code so both states dispatching and listening
share some code that is not a state function.  This is essentially a case
of code reuse where you reuse a state function from another as if it was a
helper function.  No immediate harm in that.

> 
> Does calling state functions directly break any OTP principles or lead to any strange behaviour (eg. the main gen_fsm loop not knowing which state it’s in +/- any corruption of centrally held StateData of the main server loop) ?

Well, the gen_fsm engine still thinks you are in state dispatching, but
since the code either returns {stop,,} or {next_state,,} this is of little
importance.  It will look as if you stopped from the dispatching state,
while you returned {stop,,} from the listening function, which probably
is of little importance.

But there is no centrally held state data apart from the state name that will
be corrupted.  The StateData is in the callback module's possesion at the
time so it will not be held by the gen_fsm engine until you return it.

> 
> I couldn’t find anything about direct state function calls after extensive googling.
> 
> The gen_fsm docs don’t mention anything about it either way - it only mentions state transitions via state function return tuples, and execution of state functions via matching events received while in that state.

Yes.  This is a flaw we do not like to talk about. But sufficiently
complicated state machines often end up doing this with gen_fsm.

> 
> Could someone who is familiar with Erlang/OTP give some advice ?

I just have to push for gen_statem that I created partly due to this
problem in gen_fsm.  It is a new addition to Erlang/OTP.  Some small API
changes are anticipated for the next release, but it will not be removed.
It will in the future be recommended to use instead of gen_fsm.

    http://www.erlang.org/doc/man/gen_statem.html
    http://erlang.org/doc/design_principles/statem.html

In gen_statem you can return with an action {next_event,internal,Something}
which will truly change states and send an internal event Something
to arrive before any queued events. This sounds like exactly what you
are missing from gen_fsm.

There was a lengthy and somewhat heated discussion while developing
gen_statem (in which Fred Hébert participated, by the way):

    https://github.com/erlang/otp/pull/960

There are small API changes in the pipeline probably for OTP-19.1 discussed
in this pull request:

    https://github.com/erlang/otp/pull/1133

but proposed in this pull request (currently in the lead):

    https://github.com/erlang/otp/pull/1135

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB