[erlang-questions] I Hate Unit Testing...

[Richard Carlsson]

Steve Davis wrote:
> Almost, but significantly... not quite. No compile step.

Hm, well, you've heard of this nifty thing called "make"?
But agreed, it is an additional step.

>> If you want to save the step of creating a separate file, then by
>> all means place the tests directly in the module you're writing,
> 
> ...that could be a bit of a problem for version control (and 
> code-bloat). Imagine wishing to add new tests after delivery (and the 
> customer bitching about a bug you hadn't considered) ...how would you 
> test their *same* build/codebase? All in all, test code *inside* the 
> module is a **very-big-and-bad** idea... even evil.

Yes. I generally recommend that the functions that test your exported
API are placed in a separate module, but it tends to depend on what kind
of software you are writing, how you ship it to customers, etc.

>> However, some direct eunit support for reading a comma-separated list
>> of functions and input/output values could be handy. I'll add that to
>> my feature list.
> 
> Very cool!

One thing, though: In my opinion, the best kind of tests are those that
avoid hard-coded values (apart from specific boundary values), and
instead express the properties in terms of "doing this should give
the same result as doing that", and similar. The problem with a table
(both as a sequence of copy/pasted microtests in the code, with a few
selected inputs and results, or as a file that lists the corresponding
inputs/results), is that unless you can quote a good reference document
for those values (such as a physics handbook), they have a tendency
to get modified over time, to make the tests pass, and the whys and
wherefores of those changes get quickly forgotten.

It is much more difficult to tweak a test on a whim to make it pass, if
the test just says something like "this algorithm should give the same
result as that one from the standard library", or "for a bunch of
randomly selected inputs n in the range [x,y] where x<y, the following
should be true for the value of f(n): ..." (or for all values, if
it does not take too much time). This basically means that tests
_need_ to be code (interpreteded, compiled, full Erlang or a subset,
a totally separate language, or whatever, but it must be powerful
enough to express such things), in order to be any good.

So: by deliberately sticking with the most primitive form of tests,
you are both missing out on the really _good_ tests: the ones that
take a minute or two more two write, but which will save you again
and again over time and are easy to verify even after a long time
that they are still correct (and a single such test can often
replace a dozen entries in an input/output table). Instead, you
are making yourself vulnerable to drifting test suites.

But I don't dispute your right to prefer testing via tables, and there
are some quite legitimate cases of this, where one generates the tables
automatically in a separate step (perhaps from specification documents),
so I think some built-in support could be good.

     /Richard