[erlang-questions] Testing gen_statem timers

[Adam Lindberg]

I concur with Jesper and Fred. Timer data should be injected into the process, with a sensible production default.

Personally I would keep all the different timer values in a map in the process state, seeded by a default argument in my own MyProc:star_link/X function. This map can then be merged with an optional map supplied by the user. When a timer is started in gen_statem, you would do a lookup in this map based on e.g. the timer name. For tests, you could then inject a custom map where you override some timer values to match what is desired for the test. In my opinion, no extra support in gen_statem is needed for this.

Relying on shorter but still not exactly precise timeouts in tests is still tricky. What could be useful to have support for in gen_statem is firing the timeout events manually. I.e. in a test where you set up the timer in gen_statem, there would then be a way to manually fire a timer event in a gen_statem process. That way, the test becomes very readable and deterministic. Not sure how such an API would look like though.

Cheers,
Adam

> On 16. Aug 2019, at 15:47, Raimo Niskanen <raimo+erlang-questions@REDACTED> wrote:
> 
> On Fri, Aug 16, 2019 at 12:36:29PM +0200, Torben Hoffmann wrote:
>> On Fri, 16 Aug 2019 at 10:43, Raimo Niskanen <
>> raimo+erlang-questions@REDACTED> wrote:
>> 
>>> On Thu, Aug 15, 2019 at 06:44:43PM +0200, Torben Hoffmann wrote:
>>>> Hi Micael,
>>>> 
>>>> I'm very biassed here, but apart from my creator fondness many of my
>>>> colleagues are very happy about the approach that my Chronos library
>>> takes
>>>> when it comes to timers.
>>>> 
>>>> https://github.com/lehoff/chronos
>>> 
>>> I notice that in that README.md you compare Chronos with the timeout
>>> feature
>>> in gen_server and gen_fsm.  The timers in gen_statem were specifically
>>> designed to remedy most of their shortcomings.
>>> 
>>> So, from a usability point of view, can you elaborate what Cronos
>>> can do better than gen_statem timers?
>>> 
>>> I have a todo to update the README.md to cover gen_statem as well - thanks
>> for reminding me!
> 
> Great!  No problem ;-)
> 
>> 
>> I have worked a lot with gen_statem - absolutely a very good component and
>> easy to work with.
> 
> Always nice to hear! :-)
> 
>> But the problem of mocking still remains the same.
>> When testing I prefer to mock time so that I don't have to wait for the
>> timeout to occur.
>> Furthermore, I'm so heavily influenced by my ten years at Motorola that I
>> want the timers to stand out as in all the nice SDL diagrams in the
>> standards shows.
>> That means that I want an easy way to check that a timer has been started
>> (mock the call to chronos:start_timer()) and then an easy way to induce the
>> expiry of said timer without having to wait for whatever the delay is.
>> When you have timers in the minutes range you just don't want your
>> automated tests to wait it out.
>> 
>> If there is a way to avoid this with gen_statem I'd be most interested!
> 
> Not today.
> 
> For debugging only, not mocking, it would be possible to log timer actions
> in the sys log, and to include timer statistics i.e the number of running
> timers or a list of them in sys:get_status/1.
> 
> Would that be interesting fetures?
> 
> For mocking, a possible new feature would be to make the calls to
> erlang:start_timer/4 and erlang:cancel_timer/1 pluggable, i.e call them
> through fun:s.  Then the plugged in functions may for example communicate
> with a registered timer server that can do as it pleases about when to
> fire timeouts, log timer starts and cancels, etc...
> 
> How dynamic would such a feature need to be?
> * An option to gen_statem:start{,_link}/3,4?
> * Read application variable at server start?
> * Return an action from the callback module (so the callback module
>  can implement a function to activate mocking)?
> * Other suggestions?
> 
> Interesting topic - I did not forsee this need...
> 
> BRs
> / Raimo
> 
> 
>> 
>> Cheers,
>> Torben
>> 
>> 
>>>> 
>>>> Basically, don't use the internal timers, but do read the full README
>>> from
>>>> Chronos.
>>>> 
>>>> Very happy to assist you and improve anything in Chronos, if you find it
>>>> confusing.
>>>> 
>>>> Cheers,
>>>> Torben
>>>> 
>>>> On Thu, 15 Aug 2019 at 13:35, Micael Nussbaumer <
>>> micaelnussbaumer@REDACTED>
>>>> wrote:
>>>> 
>>>>> Hi, first time poster (but I've read many threads in here).
>>>>> 
>>>>> How does one usually test gen_statem internal timer events?
>>>>> Usually you test the effects of the behaviour somehow, and I can test
>>>>> that. But in this particular case I had a situation where timers with
>>> uniq
>>>>> references are created, later on they usually get removed and all is
>>> fine -
>>>>> they're created for each extra worker started and at some point the
>>> worker
>>>>> is removed by the timer event that trigger checking some conditions.
>>>>> 
>>>>> I noticed after that, the way I had written it, if an extra worker was
>>>>> started then died for some other reason than the timer event taking it
>>> out,
>>>>> I could theoretically accumulate timers that wouldn't be removed, I've
>>>>> since then corrected that but still I have no tests ensuring me that it
>>>>> actually works.
>>>>> So my question is if there's any even way to test it (even if with
>>>>> tracing)?
>>>>> Thanks
>>>>> 
>>>>> *M*icael *N*ussbaumer
>>>>> artistic portfolio <http://cargocollective.com/micaelnussbaumer>
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> http://www.linkedin.com/in/torbenhoffmann
>>>> @LeHoff
>>> 
>>> 
>>> --
>>> 
>>> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>> 
>> 
>> 
>> -- 
>> http://www.linkedin.com/in/torbenhoffmann
>> @LeHoff
> 
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 
> 
> -- 
> 
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions