[erlang-questions] Increase quality by tagging release candidates?

[Loïc Hoguin]

On 11/29/2012 09:42 PM, Björn-Egil Dahlberg wrote:
> Hi there!
>
> I'm curious; would pushing and tagging release candidates to GitHub for
> user testing be worthwhile?
>
> We already do this for major-versions, i.e. R14A, R15A and soonish R16A,
> and we get some useful information from this. I want to encourage people
> to test our otp HEAD at GitHub when nearing a release. Especially those
> we say are release-candidates, i.e. R*A-releases. I also think we should
> start tagging those as release candidates, i.e. R15A-RC1, R15A-RC2, etc,
> further making the point of a release candidate so we don't have users
> taking a release candidate for production .. or include it in a distro.
>
> Now, should we also do this for maint-releases also? At Erlang/OTP we
> have *extensive* testing on a multitude of operating systems,
> architectures and configurations. We can however, always test even more. =)
>
> Normally we have "code-freeze" a couple of weeks before a release and
> then a stabilization phase to iron out, hopefully, the last few glitches.
>
> I'm thinking, at code-freeze - push what we have, tag it and notify you
> via this or other medias that we have a src-state that contains more
> signal than noise and we wish that you test it against your product and
> your testcases. Hopefully, you could get information back to us by
> saying *thumbs up* or more importantly saying "I'm concerned about
> this behavior here .." if you encounter something worrisome.
>
> The window to do changes are small at that point and we would certainly
> only correct things that are apparent bugs - correcting one thing runs
> the risk of creating even bigger, unforeseen, problems. Ultimately some
> sort of risk/reward decision will be taken by us.
>
> Is this of intrest? Should we start tagging release-candidates? Would
> you be interested to test release candidates on your products and let us
> know problems before a release?
>
> I think it would lead to even better product quality.

Me too, for all the reasons you mentioned. Tag when it's code freeze, 
inform people about it, and the concerned citizens will test the code 
against their projects. Everybody wins.

-- 
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu