[erlang-questions] Fwd: FW: Breaking backwards compatibility in Release 17.0-rc2

Thu Mar 6 11:15:46 CET 2014

Admittedly, we did not communicate those (intentional) incompatibilities
effectively prior to the release. Hence, we will make some adaptations to
ease the transition. They will be published in the master branch of the
erlang/otp GitHub repository at the beginning of next week.

Re: 1. utf-8 is now the default encoding

The default encoding of Erlang files has been changed from ISO-8859-1
(Latin-1) to UTF-8. The encoding of XML files has also been changed to
UTF-8.

In OTP 17.0-rc{1,2}, a file that is encoded in latin-1 and contains
non-UTF-8/non-ASCII-7 characters, causes a compiler error similar to the
following:

  tst.erl:1: cannot parse file, giving up
  tst.erl:1: no module definition
  tst.erl:1: cannot translate from UTF-8

In OTP 17.0, if a file is encoded in latin-1 and contains
non-UTF-8/non-ASCII characters, but does not declare the encoding with a
magic encoding comment at the beginning of the file, epp (the Erlang code
pre-processor) issues a deprecation warning, and processes the file again,
assuming latin-1 encoding.

In a future major version, preferably in OTP 18, the deprecation warning
will be turned into an error again. That is, only UTF-8 encoded files, and
files that declare the source code encoding at the beginning of the source
code file, will be accepted.

NOTE: The deprecation workaround in OTP 17 does only apply to Erlang source
code files, but not to other files; e.g., yecc files and files read by
file:consult/1.

For more information, see the STDLIB User's Guide, 2 Using Unicode in
Erlang [1] and the STDLIB Reference Manual, epp [2].

[1] http://www.erlang.org/doc/apps/stdlib/unicode_usage.html
[2] http://www.erlang.org/doc/man/epp.html

Re: 2. Dialyzer dislikes queue(), dict(), ...

OTP-10342
The pre-defined types array/0, dict/0, digraph/0, gb_set/0, gb_tree/0,
queue/0, set/0, and tid/0 have been deprecated. They will be removed in
Erlang/OTP 18.0.

Instead the types array:array/0, dict:dict/0, digraph:graph/0,
gb_set:set/0, gb_tree:tree/0, queue:queue/0, sets:set/0, and ets:tid/0 can
be used. (Note: it has always been necessary to use ets:tid/0.)

It is allowed in Erlang/OTP 17 to locally re-define the types array/0,
dict/0, and so on.

New types array:array/1, dict:dict/2, gb_sets:set/1, gb_trees:tree/2,
queue:queue/1, and sets:set/1 have been added.

A compiler option, nowarn_deprecated_type, has been introduced. By
including the attribute

    -compile(nowarn_deprecated_type).

in an Erlang source file, warnings about deprecated types can be avoided in
Erlang/OTP 17.0.

The option can also be given as a compiler flag:

    erlc +nowarn_deprecated_type file.erl

Andreas Schumacher, Erlang/OTP, Ericsson AB

---------- Forwarded message ----------
From: Andreas Schumacher <andreas.schumacher@REDACTED>
Date: Thu, Mar 6, 2014 at 9:56 AM
Subject: FW: [erlang-questions] Breaking backwards compatibility in Release
17.0-rc2
To: "andreasschu@REDACTED" <andreasschu@REDACTED>

*From:* erlang-questions-bounces@REDACTED [mailto:
erlang-questions-bounces@REDACTED] *On Behalf Of *Jesper Louis Andersen
*Sent:* den 28 februari 2014 00:07
*To:* Erlang (E-mail)
*Subject:* [erlang-questions] Breaking backwards compatibility in Release
17.0-rc2

Release 17.0 brings two changes which prove to take some work getting
around.

1. utf-8 is now the default encoding.

This is a rather insignificant change. The source code which uses latin1
can be fixed by one of three ways:

* Tell the compiler the file is latin1. This won't work going forward but
works now.

* Change the file to utf-8. This won't work going backward a long way. But
it will work going backwards for a bit.

* Change the file to ASCII. This works both backward and forward as long as
we want.

This is a benign problem. I have tried compiling some projects and it turns
out there are numerous repositories which needs fixing now. But the fix is
rather simple.

2. Dialyzer dislikes queue(), dict(), ...

Dialyzer now prefers using queue:queue() and the like. This is *definitely*
the right thing to support as it is much more consistent with the rest of
the system and doesn't treat certain types as magically introduced types.

-module(z).

-export([f/1]).

-spec f(queue:queue()) -> queue:queue().

f(Q) -> queue:in(3, Q).

Which is nice, but this doesn't work on R16B03:

z.erl:5: referring to built-in type queue as a remote type; please take out
the module name

z.erl:5: referring to built-in type queue as a remote type; please take out
the module name

So here, I have no way of getting my source code to work with both R16 and
17.0 easily. There is no transition period so-to-speak. Many projects run
with warnings-as-errors and they are in trouble:

* They can't compile

* They can remove the warnings-as-errors but this defeats the purpose

* They will have warnings spewed out over the console all the time

In the case of crypto:hash/2, we had somewhat the same situation. Prominent
projects like Yaws, and lesser projects like Emysql has EPP macros in place
as well as detection in order to figure out what to do. Or you can disable
the warnings in this case specifically for this. But can I do the same with
wrong type specs? Also, this workaround is done in almost every project out
there, which is darn irritating.

I don't know what we need to solve this. At one point, I would really like
to have a set of feature flags

http://www.lispworks.com/documentation/HyperSpec/Body/v_featur.htm , ZFS,
...

where you have a way to compile-time scrutinize what your environment
supports. Another way to solve it is the variant Go uses, namely "build
constraints"

http://golang.org/pkg/go/build/#pkg-overview

which will mention under which circumstances to include a file as a part of
an application. This would allow for easy handling of crypto:hash/2, but I
do note it will fail on the dialyzer problem. It looks like the only sane
way to solve that is to allow both queue() and queue:queue() as aliases for
a major release and then proceed to remove queue().

Am I completely wrong here? I can accept languages evolve and that Release
17 has maps which will be used and break a lot of software for R16 quickly.
But I also feel we should have some way of having a process so there is a
way to handle this gracefully going forward. It is natural for libraries
and languages to evolve and break compatibility. Yet, it should be easy to
handle for programmers. There is much time wasted, which could be used
better were there a nice solution.

Thoughts?

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140306/3646eefa/attachment.htm>