[erlang-questions] Breaking backwards compatibility in Release 17.0-rc2

[Loïc Hoguin]

My plans are the following:

*master* officially supports only R<N>B and R<N-1>B for building. If 
supporting R<N-1>B is too annoying (for example R14 vs R15 -callback), 
then R<N-1>B support is quickly dropped from master. (Of course, if 
master can support even R<N-2>B, then it is supported.)

*R<X>B branches* are branches created for older versions of Erlang that 
were dropped from master. Two scenarios may happen.

  *  If making master compatible with that version of Erlang can be 
automated with a small script (a few regexpes, for example similar to 
s/binary_to_integer/list_to_integer(binary_to_list/g), then they can 
continue to follow master and receive new features. When releasing a new 
version, a special tag would be created with the Erlang version clearly 
indicated, for example 1.2.3-R16.

  *  Otherwise, it's handled like an API breaking version and only 
important bug fixes get backported.

Before tagging a new release, you simply have to run tests across all 
versions (which you should do anyway). If building fails then, a new 
branch is required. Branches need to be updated for the upcoming release 
and tests ran for them too. It can almost all be automated.

People who follow master understand that things may break already, so 
they are not a concern.

The good thing of all that is that you can now always write code using 
the latest features of the VM, and only have to worry about older Erlang 
releases when you increment your version.

The bad thing is that the code for older Erlang releases might not be as 
Dialyzer friendly or as optimal as it could be. But at least it compiles.

None of this requires any change in how the OTP team currently does 
things, and it is completely future proof.

On 02/28/2014 12:06 AM, Jesper Louis Andersen wrote:
> Release 17.0 brings two changes which prove to take some work getting
> around.
>
> 1. utf-8 is now the default encoding.
>
> This is a rather insignificant change. The source code which uses latin1
> can be fixed by one of three ways:
>
> * Tell the compiler the file is latin1. This won't work going forward
> but works now.
> * Change the file to utf-8. This won't work going backward a long way.
> But it will work going backwards for a bit.
> * Change the file to ASCII. This works both backward and forward as long
> as we want.
>
> This is a benign problem. I have tried compiling some projects and it
> turns out there are numerous repositories which needs fixing now. But
> the fix is rather simple.
>
> 2. Dialyzer dislikes queue(), dict(), ...
>
> Dialyzer now prefers using queue:queue() and the like. This is
> *definitely* the right thing to support as it is much more consistent
> with the rest of the system and doesn't treat certain types as magically
> introduced types.
>
> -module(z).
>
> -export([f/1]).
>
> -spec f(queue:queue()) -> queue:queue().
> f(Q) -> queue:in(3, Q).
>
> Which is nice, but this doesn't work on R16B03:
>
> z.erl:5: referring to built-in type queue as a remote type; please take
> out the module name
> z.erl:5: referring to built-in type queue as a remote type; please take
> out the module name
>
> So here, I have no way of getting my source code to work with both R16
> and 17.0 easily. There is no transition period so-to-speak. Many
> projects run with warnings-as-errors and they are in trouble:
>
> * They can't compile
> * They can remove the warnings-as-errors but this defeats the purpose
> * They will have warnings spewed out over the console all the time
>
> In the case of crypto:hash/2, we had somewhat the same situation.
> Prominent projects like Yaws, and lesser projects like Emysql has EPP
> macros in place as well as detection in order to figure out what to do.
> Or you can disable the warnings in this case specifically for this. But
> can I do the same with wrong type specs? Also, this workaround is done
> in almost every project out there, which is darn irritating.
>
> I don't know what we need to solve this. At one point, I would really
> like to have a set of feature flags
>
> http://www.lispworks.com/documentation/HyperSpec/Body/v_featur.htm ,
> ZFS, ...
>
> where you have a way to compile-time scrutinize what your environment
> supports. Another way to solve it is the variant Go uses, namely "build
> constraints"
>
> http://golang.org/pkg/go/build/#pkg-overview
>
> which will mention under which circumstances to include a file as a part
> of an application. This would allow for easy handling of crypto:hash/2,
> but I do note it will fail on the dialyzer problem. It looks like the
> only sane way to solve that is to allow both queue() and queue:queue()
> as aliases for a major release and then proceed to remove queue().
>
> Am I completely wrong here? I can accept languages evolve and that
> Release 17 has maps which will be used and break a lot of software for
> R16 quickly. But I also feel we should have some way of having a process
> so there is a way to handle this gracefully going forward. It is natural
> for libraries and languages to evolve and break compatibility. Yet, it
> should be easy to handle for programmers. There is much time wasted,
> which could be used better were there a nice solution.
>
> Thoughts?
>
> --
> J.
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

-- 
Loïc Hoguin
http://ninenines.eu