[erlang-questions] erlang property testing

[James Churchman]

The other thread seems totally off topic to me, missing the wider point of "testing in erlang" so I thought I would start a second that does not contain intimacies the gpl, however fascinating and leave that for the other thread.

As I see it :

] In the near past there was eunit etc.. but it has limitations
] QuickCheck came along and shines a "new light" on the world of testing
Unfortunately it's closed source and costs a lot of money, this is great for the likes of big businesses creating custom software :
They pay in the big picture a small amount of money, get fantastic support, find bugs quickly; it's proven the quicker you find a bug the it costs dramatically less
For opensource projects it was a problem; anyone with out a licence could not even test the code or run the suite, so cant test their pull requests first
For startups with limited cash, it was a problem
For people something that is not 100 % vanilla OTP/Erlang this is a problem as its suppled as complied beam code.. (eg for testing erjang compiler etc..  etc..  etc.. ) a big problems for a "testing tool"
For people first investigating and learning erlang it was a problem
For people with a want to extend its functionality etc.. it was a problem; it was closed source.. all the amazing minds in the erlang world were locked out of making improvements they saw fit
] PropEr (unannounced) arrives on the scene, is 0 cost and the source is available to tinker with
Modifications to the source can be made
It also comes with the benefit that it can test the specs** ( should be easy deriving the type generators from the spec AST tho)  
] quickcheck mini comes out, a bit late and with less functionality than the other two and it's still closed source, but is available for 0 cost

This is roughly the situation we are right now and there seems a few problems:

a) QuickCheck is not really compatible with PropEr
This seems to be a problem, and often choice is good. You can pick the tool that suites your needs.. eg mochiweb or yaws, totally different really. In this case however its just plain incompatibility and bad.
It means that if i start on PropEr, and decide i want someone else to build me a more intensive test suite; this seems a good idea.. if someone else tests your code.. specially someone with more experience; All tests have to be modified and moved over
In many cases there will be two camps, the PropEr and the QQ camp. Over time the two will divider further.
b) The QuickCheck product and the ideas behind property based testing come from QuviQ and its employes.
I think that everybody own them a large degree of gratitude, tho i also think that their business naivety does show threw here.
It was just 100 % obvious to I think me and many other erlang dev's that a tool *this* good and *this* powerful was always going to be cloned, and the clone was never going to be 100 % compatible. Whether it was PropE or Triq or one written by someone else this was always going to happen & happen quickly. Also the majority of the money is charging the big companies, with ALL the money, for custom service.
I think that everybody appreciates the presence of QuviQ & co in the erlang community, but there was a desire and plain need for a better unit testing tool that can be modified, used by anybody with out licence etc..
c) A testing tool is very different. I think the gpl has strengths, in that it stops a huge company grabbing your code and selling a better version, and in many ways it protects the original authors, but it also comes at great cost. A testing tool is by its very nature something to test your code, it needs to be "embedded" in your product if it to do anything but test the IO of the entire system. In this case the gpl is a poor fit.. PropEr will never be the product it intended to be whilst under the GPL... in many ways its easier to embed beam code for an entirely closed testing tool.

So the obvious questions is where does this near future train crash end up?

In my dream land, the original QQ goes MIT licence and we all just contribute to that, the obvious problem I see is that it's lightly not the business plan that QuviQ ever dreamed of. Tho lightly neither was the arrival of clone of QQ , which will lightly be far more damaging to their biz as these clones are not fully compatible. Then PropEr can be seen in the future as an instigator of great change, but not really used

The other options are :

PropEr Moves to a more sensible "for a testing tool " licence than the gpl. As the other thread reveals this is going to be a recurring nightmare for ever on, until of course a non gpl one arrives. 
Then possibly, if the entire erlang testing ecosystem, including maybe larges sections of OTP move to PropEr, then it may become untenable to have a closed alternative. maybe in this case, as unusual as it may seem QuviQ may want to offer commercial support for PropEr.
If it was me in that position personally, I would be rubbing my hands together. The thought that an entire industry may take up my idea would be great, as the saying goes it's usually better to have a small % of something huge than a large % of something tiny, especially when it's your expert domain that goes mainstream. I think its just more complex biz model than selling.

A legal & tecnical workaround.
This is my least favroable option, but if there is an intermediary that can run either QQ or PropEr that is dual licences then its licence Ok and API differences ok. The problem with this is its extra work and ends up with the usual "least common denominator". If we liked the solution we would all be writing java.

We all finish Triq.
I am fairly confidant i could spin up the spec testing quite fast. I understand the spec AST's for transforming into something else I use. Maybe a few others could get the shrinking code more up-to-speed. State machines etc.. can come later. I think this is very feasible.

A few more I invite people to propose. I also welcome replies that are less a minutiae of a minutiae but more *broad* in nature of what is the best end goal.

Sincere apologies if this trampled on a whole heap of clever and wise peoples toes, but better in the clear now i feel than have undisclosed problems in the future

J