<div dir="ltr">This brings the question, is there a library for writing temporal properties for testing with quickcheck or proper? I mean properties that say something about the sequence of trace messages and not about the contents of the list of trace mssages for example.<br></div><br><div class="gmail_quote"><div dir="ltr">Em sex, 31 de ago de 2018 às 16:24, Jesper Louis Andersen <<a href="mailto:jesper.louis.andersen@gmail.com">jesper.louis.andersen@gmail.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I've used the sys module more than once, in both debugging and testing. It is useful, also because you can use sys:log/2,3 to have a trace log of a running process and inspect it later.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Another approach I've used is to add trace messages to a system. A module defines this function:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">report_event(_DetailLevel, _From, _To, _Label, _Contents) -> hopefully_traced.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">And you call it in your system. Now, you can attach a tracer to this call in tests and verify the trace you get is what you expect. This allows you to "see inside" your system and verify its behavior. More advanced solutions involves rewriting traces into the notion of a temporal monad and then asking questions on the trace for correctness.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Yet another approach is to rewrite your code into a computational (pure) part which sets up a set of actions but has no effect, and an effectful part which runs said actions. This allows you to scrutinize the actions without having to bind your system up. Look up, e.g., the Elm architecture for this approach. The basic idea is that of a free monad: rather than running the program, return a command set which can be interpreted either as a test (where the effects are logged) or as a real system (where the effects are run). We do this at $WORK, which means we can test all of our stack without ever having to run the system as a whole in unit tests. In fact, we don't even need a database, or supervisor tree, since we just simulate the universe around a process. Integration tests are run later via a QuickCheck like tool written by Martin Gausby - treating the system as a black box and poking it from the outside to verify we can glue things together as expected.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Aug 31, 2018 at 7:07 PM Rodrigo Stevaux <<a href="mailto:roehst@gmail.com" target="_blank">roehst@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>yes, we fundamentally are trying to do integration testing.<br></div><div><br></div><div>unit testing process communication pairwise by sending a message to the parent process is something we tried to do and it worked reasonably well. but we want to know that the whole composition of processes works well.<br></div><div><br></div><div>the challenge we found on that approach is that the processes have names and we would have to find a way to inject the pid of the "next" process in the chain and make it the test process during testing and other process during production (which can be done with a little bit of instrumentation and conditional compiling -- we are on Elixir actually).</div><div><br></div><div>is using the "sys" module a regular practice?<br></div><div><br></div><div>thanks for the reply.<br></div></div><br><div class="gmail_quote"><div dir="ltr">Em qui, 30 de ago de 2018 às 20:33, Fred Hebert <<a href="mailto:mononcqc@ferd.ca" target="_blank">mononcqc@ferd.ca</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 08/30, Rodrigo Stevaux wrote:<br>
>Message comes in, logic applies, message goes out.<br>
>The generic servers are very easy to test in a pure fashion: just test the<br>
>return values of handle_call for a given state.<br>
><br>
<br>
That can be done as a regular unit test, yes.<br>
<br>
>But:<br>
>- unit testing should test that the unit under test forwards a message to a<br>
>given address (and that is not reflected on the handle_call return value)<br>
<br>
Aside from using mocks to know that the proper function calls have taken <br>
place, you can't easily test the forwarding of messages since that would <br>
imply side-effects turning your unit tests into integration tests.<br>
<br>
>- doing the integration testing of a "pipeline" of process involves testing<br>
>that ultimately a message gets to the last process.<br>
><br>
<br>
Yes and no. You can test that each element of the chain properly <br>
forwards data to the right destination. You can do that if you make the <br>
endpoint/address/destination for each forwarded message parametrizable <br>
or configurable. Forward the data to the test process instead to confirm <br>
that the right side-effect has taken place.<br>
<br>
This lets you test each section of the chain in isolation; then if all <br>
sections are well-behaved, you can infer that the whole chain should <br>
behave well. You can confirm that with a more naive end-to-end test, <br>
knowing that individual components are tested more thoroughly.<br>
<br>
>Given the maturity of Erlang, I'd expect to promptly find solutions to this<br>
>case.<br>
<br>
On top of the techniques above, you can try using the `sys' module to <br>
introspect all OTP behaviours and look at their transitions as a neutral <br>
observer:<br>
<br>
1> sys:get_status(disk_log_server).<br>
{status,<0.75.0>,<br>
{module,gen_server},<br>
[[{<0.76.0>,'$#group_history'},<br>
{'$initial_call',{disk_log_server,init,1}},<br>
{'$ancestors',[kernel_safe_sup,kernel_sup,<0.45.0>]}],<br>
running,<0.66.0>,[],<br>
[{header,"Status for generic server disk_log_server"},<br>
{data,[{"Status",running},<br>
{"Parent",<0.66.0>},<br>
{"Logged events",[]}]},<br>
{data,[{"State",{state,[]}}]}]]}<br>
2> P = self(),<br>
2> sys:install(<br>
2> disk_log_server,<br>
2> {fun(_Acc, Event, State) -> P ! {trace, Event, State} end,<br>
2> nodata}<br>
2> ).<br>
ok<br>
3> disk_log_server ! fake_data.<br>
fake_data<br>
4> flush().<br>
Shell got {trace,{in,fake_data},disk_log_server}<br>
Shell got {trace,{noreply,{state,[]}},disk_log_server}<br>
ok<br>
<br>
Those won't let you see arbitrary calls being made, but if you have a <br>
process chain A -> B -> C, then you can, by looking at the traces of all <br>
three processes, track the history of a call and the consequences they <br>
have.<br>
<br>
The sys module is full of really cool functionality few people know <br>
about, and that can make live debugging, but also testing much simpler.<br>
<br>
><br>
>I haven't found a nice solution yet, so I suppose I am using OTP in a wrong<br>
>way (maybe it's not a good fit for my project), or I am choosing the wrong<br>
>architecture, or that I am missing key knowledge about how to test things<br>
>in Erlang.<br>
<br>
There aren't enough details to know at this point. I frequently just use <br>
`meck' (<a href="https://github.com/eproxus/meck" rel="noreferrer" target="_blank">https://github.com/eproxus/meck</a>) as a library, which lets you <br>
replace arbitrary function calls through hot code loadig, get traces and <br>
counts of all calls and values returned of unmodified code, and so on.<br>
<br>
If what you're looking for is a way to track side-effects in a very <br>
whitebox manner, meck is hard to beat.<br>
<br>
><br>
>Can anyone shed some light on this questions? Thanks in advance!<br>
<br>
please provide more details about the nature of the side-effects you are <br>
trying to observe in a unit test. My default reflex would be to say that <br>
the moment you're testing side-effects, you are doing integration <br>
testing.<br>
<br>
Regards,<br>
Fred.<br>
</blockquote></div>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_5050806217472718227gmail_signature" data-smartmail="gmail_signature">J.</div>
</blockquote></div>