[erlang-questions] EUnit quirks

[Gareth Adams]

Hi,

This reply is to both you, Richard, and Daniel (didn’t want to pollute the list with two answers).

Regarding 1:

I’d worked out that I could wrap up the setup so it doesn’t blow and cleans up after itself if there’s an error, I’d just hoped that eunit could have done that for me by always running the cleanup function.
I like the idea of spawning the tests – I’ll try that in work tomorrow and see what happens.

I suppose the setup making sure that the system is in a known state before it starts is not a bad idea either but it may lead to lazy tests where no one cleans up after themselves.

Regarding 2:

I’ve written a quick test where the first two tests are not run because I’ve called an unknown function but the third test is run. As I say, the console does report the failing test but when we’ve got ~1000 tests that often produce debug printouts this gets lost. I’ve noticed the “One or more tests were cancelled” – I don’t recall seeing that in work, I’ll check. However, we use intellij in work which doesn’t display the console results in the test runner by default.

-module(aborted).

-include_lib("eunit/include/eunit.hrl").

aborted_test_() ->
    {
        foreach,
        fun setup/0,
        fun cleanup/1,
        [
            fun test1/1,
            fun test2/1
        ]
    }.

setup() ->
    state.

cleanup(_State) ->
    ok.

test1(State) ->
    call:unknown(State),
    [
        {"Passing test", ?_assertEqual(state, State)},
        {"First Failing test", ?_assertEqual(not_state, State)}
    ].

test2(State) ->
    [
        {"Second Failing test", ?_assertEqual(really_not_state, State)}
    ].


Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Richard Carlsson<mailto:carlsson.richard@REDACTED>
Sent: 16 July 2017 10:51
To: Gareth Adams<mailto:gareth.adams@REDACTED>
Cc: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>
Subject: Re: [erlang-questions] EUnit quirks

1: A setup function is assumed to either succeed, with a possibly modified world as result, or fail, with an unchanged world as result. If you need to make multiple changes, the idea is that you either write separate setup/cleanup functions for each change and nest them to create a complete setup, or you write a single setup function that does its own internal catching of errors and partial cleanup.

2. All exceptions in a test case should be handled the same by eunit. Asserts are just one way of throwing an exception. Do you have an example to demonstrate what you are seeing?

        /Richard

2017-07-14 18:45 GMT+02:00 Gareth Adams <gareth.adams@REDACTED<mailto:gareth.adams@REDACTED>>:
Hi,

I've been writing some eunit tests using fixtures (http://erlang.org/doc/apps/eunit/chapter.html#Fixtures) and have noticed a couple of quirks.

1. If the setup fails for some reason cleanup is not called. This can be an issue if you're mecking things as the don't get unmecked so the next test setup fails etc. I find myself lost in a cascade of failures, not knowing which tests are really failing.

2. If something fails in the test itself that's not part of an assert, eunit cancels the tests and they are not run. Although this is reported in the console, if you have a lot of tests with debug statements this can be lost. The only thing that indicates that they've not been run is the number of tests decreases, which  au not be noticed.

Has anyone else had these issues? If so, how did you handle them?

Thanks
Gareth

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>
http://erlang.org/mailman/listinfo/erlang-questions


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170716/e6688eb6/attachment.htm>