[erlang-questions] Do you know what your code is doing at night?

[Fred Hebert]

I pretty much agree with Loïc here.

I'd say standard tests will still make sense for:

- working around when finding a good property is really hard and/or 
  taking time, especially if you're on a tighter schedule, even more 
  likely to happen as a beginner.
- To augment a property. For example, the classic 'decode(encode(X)) == 
  X' case is made stronger if you have ways to say what the encoded 
  format should look like as well. Regular tests can work as 'anchors' 
  that cover some edges that are hard to do as a property
- To store counter-examples of past failures as regression suites
- To write some explicit examples of your code to ensure it works (as 
  Loïc mentioned, RFCs are a good source of that)
- As a way to get started writing your properties (write a few examples, 
  and *then* generalize them into a property)

The anchoring bit is a kind of important one, especially since if you're 
not careful, it's easy to over-estimate the quality of property test.  
Usual tricks to help there (aside from anchoring with regular tests) 
will be to:

- look at coverage
- look at the statistics of the distribution of the generators
- do negative testing of your tests (break the code on purpose in a 
  subtle way, make sure one property finds the error)

So the TL:DR; would probably be that a bunch of unit tests are 
complementary, but a bunch of them can be made redundant as well.

On 07/13, Loïc Hoguin wrote:
>On 07/13/2017 09:12 AM, Manfred Lotz wrote:
>>However, I have a general question:
>>
>>Are unit testing and property based testing to be seen as both
>>complementing each other, or is property based testing to be regarded
>>to fully replace unit testing as it should (?) cover a proper superset
>>of the tests in unit testing?
>I'm not Fred, but I do have some insight on that.
>
>Property based testing can largely replace the few random unit tests 
>you would write to test cases that you think about. It will also test 
>cases that you will not think about, and that's good.
>
>Unit testing is still useful to make sure the textbook cases still 
>work though. If you have a spec (like an RFC) with some examples, you 
>will want to write unit tests for those, because properties are code 
>and might not be perfect.
>
>And similarly, unit testing is still useful for testing against 
>regressions. Even if you fix your property, you should write a unit 
>test with the previously failing case to make sure you don't break 
>your code and your property again.
>
>-- 
>Loïc Hoguin
>https://ninenines.eu
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions