[erlang-questions] EUnit quirks

[Dániel Szoboszlay]

Hi,

1:
This specific problem with meck could be worked around by wrapping your
fixture into a spawn: meck (by default) unloads mocked modules when the
process creating the mocks dies.

There's also a more generic approach to this problem. I've seen two
different strategies for writing test setup/cleanup code:

   1. Setup assumes you start from a known state of the system, and it only
   has to change things that your test needs in addition to that known initial
   state. It's the cleanup's responsibility to return to the known state by
   undoing whatever the setup and the actual test could have changed.
   2. Setup doesn't assume anything about the initial state of the system:
   it has the responsibility of getting it from an arbitrary state to a known
   state that is suitable for running the test. Cleanup isn't really needed in
   this model: the next test's setup has to deal with whatever state you leave
   behind your test.

The second strategy is usually more robust, because it is very easy to
accidentally start testing when your system is not in it's assumed initial
state (at least on your own machine during development). On the other hand,
writing setup code that can take you from literally any (possibly broken)
state of the system to a known good start state is very hard. But in case
of this particular problem it's easy: just call meck:unload() from the
setup function as the first thing.

2:
Do you have a minimal example showing this behaviour? I tried to reproduce,
but at least for me eunit:test/1 does correctly report a test failed
outside of an assert as a failure too, and the return value reveals that
there was a problem as well.

Regards,
Daniel

On Fri, 14 Jul 2017 at 19:16 Gareth Adams <gareth.adams@REDACTED> wrote:

> Hi,
>
> I've been writing some eunit tests using fixtures (
> http://erlang.org/doc/apps/eunit/chapter.html#Fixtures) and have noticed
> a couple of quirks.
>
> 1. If the setup fails for some reason cleanup is not called. This can be
> an issue if you're mecking things as the don't get unmecked so the next
> test setup fails etc. I find myself lost in a cascade of failures, not
> knowing which tests are really failing.
>
> 2. If something fails in the test itself that's not part of an assert,
> eunit cancels the tests and they are not run. Although this is reported in
> the console, if you have a lot of tests with debug statements this can be
> lost. The only thing that indicates that they've not been run is the number
> of tests decreases, which  au not be noticed.
>
> Has anyone else had these issues? If so, how did you handle them?
>
> Thanks
>
> Gareth
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170715/96a69ec4/attachment.htm>