On the use of Core Erlang vs Abstract Syntax by tools after 23.0

[Björn Gustavsson]

On Mon, Jun 8, 2020 at 3:59 PM Joe Harrison <joe@REDACTED> wrote:

> Like Kostis noted in issue #1277, the fact that an EEP led to a breaking
> change in Core Erlang is surprising. The "Enhancement" in "Erlang
> Enhancement Proposal" has connotations of forwards-compatible changes in
> my mind. Perhaps I should re-evaluate my assumptions on this.

An EEP is required to contain a section about backwards compatibility.
Usually it is quite short, only stating that there are no backward
compatibility issues. Some EEPs may introduce incompatibilities out of
necessity. One example is EEP 36 that introduced line numbers in
exception.

> The only reason I know in advance that my Core Erlang analyses will not
> work on OTP 23 is because of your email.

The next time I'll do changes like this I will be sure to mention
potential incompatibilities in more places than just the EEP.

> It seems strange that low-level primops have replaced high-level receive
> expressions and pattern matches in Core Erlang. Perhaps the Core Erlang
> specification and compiler passes could have been amended, instead.
> I note that EEP-52 mentions this exact approach as a potential avenue to
> implementation but that it could have significantly complicated some
> compiler passes.

Amending the Core Erlang specification would mean that tools that
handles Core Erlang would have to be updated anyway. Whether that
would have been harder or easier depends on the tool.

> I realise however that it is somewhat poor form to criticise changes to
> Erlang/OTP when I have been given a fair chance to comment on them via
> the EEP process. Björn had to make a decision about where changes were
> to be made. With compiler complexity and the Core Erlang spec in the
> room, and without any objections from the community, Core Erlang drew
> the short straw. The change to Core Erlang could of course be reverted
> and the necessary changes could be made to the core compiler passes and
> the syntax of binary patterns in the spec, but I doubt that Björn or
> anyone else is willing to undo all of the work in PR2521 and then redo
> it the "other way" (5547 lines changed over 14 files).

Correct. Also see my comment to previous paragraph.

> It seems like the writing has been on the wall for Core Erlang for a
> little while now. As Björn noted in the comments of issue #1277, the
> biggest non-Erlang users of the BEAM target the Abstract Format instead.
> For example, the Elixir maintainers have already started working on
> implementing new features from 23.0. Perhaps they can do this easily
> precisely because they target the Abstract Format? It seems stable enough.

Yes, the Abstract Format is stable and is documented. We take pains to
preserve backward compatibility. We generally don't change it and only
extend it when the Erlang language evolves.

> At this point I'm inclined to stop using Core Erlang altogether in
> favour of the SSA representations introduced relatively recently. The
> SSA forms don't have patterns or receive expressions either, but at
> least the language has less surface area.

Yes, the SSA format is very nice, but note that we don't give any
guarantees to keep it stable. You are welcome to use it, but you must
be prepared to keep updating your use of it. For example, in OTP 24 we
might move fun lifting from the v3_kernel pass down to a SSA pass. The
main reason for that change is to eliminate or reduce the use of the
Kernel Erlang intermediate format (which is a historical artifact and
poorly understood).

> I would definitely like to see a renewed effort towards a well-defined
> and well-maintained intermediate format, but there are important
> questions to ask: who would really use it, and would the number of users
> justify the cost of maintaining it?

I agree. Such format would be nice, but defining the format and
maintain it will require a significant amount of time. I doubt that
there are enough users to justify the effort.

/Bjorn

-- 
Björn Gustavsson, Erlang/OTP, Ericsson AB