[erlang-questions] Getting rid of the preprocessor

[Vlad Dumitrescu]

Hi Richard,

My real question got buried in the discussion about my particular use
case: previously when the preprocessor and its demise were discussed,
one main reason for not being able to do anything about it at the
moment was that first we need to do something about the records. But
the preprocessor doesn't do anything to records, they are processed by
the compiler. To quote you from a recent thread: "Abstract patterns
and frames are all part of a long-time project to make the
preprocessor unnecessary". Could you please enlighten me as to how the
preprocessor is involved here?

On Fri, May 25, 2012 at 12:16 AM, Richard O'Keefe <ok@REDACTED> wrote:
> On 25/05/2012, at 12:02 AM, Vlad Dumitrescu wrote:
>> As one example, how would you parse the following line?
>>    ?HELLO(world)
> I am no friend of the preprocessor.
> But this at least is no new problem:  C has exactly the same
> problem, and presumably the C and C++ support in Eclipse
> already does something with it.

Yes, it's not new and yes, the C/C++ support does something with it.
The problem at hand is that we don't have the kind of resources that
were put into the C support by the likes of Intel, IBM, Texas
Instruments and, yes, even Ericsson. There are many tens of man-years
invested in that project...

> I presume it goes something like this:
>        if it looks like a variable, display it as one;
>        if it looks like a function call, display it as one;
>        if the display is wrong, it's the programmer's fault
>        for defining such a stupid macro.

The displaying of fancy colors in the editor is not the important
issue. An IDE needs to know as much detail as possible about the code,
to support navigation and refactorings, to be able to suggest
meaningful completions and offer reasonable ways to fix any issues it
finds. This isn't possible if one can't understand (=parse) the code
properly.

>        if the display is wrong, it's the programmer's fault
>        for defining such a stupid macro.

Unfortunately, this argument doesn't really work for legacy systems,
where the development environment has to work with code that won't be
changed unless it's broken at runtime.

For macros that expand to well-formed expressions, we can treat them
as function calls, that's the easy part. There are other uses where
the parser needs to be prepared for all kinds of weirdness. The most
pervasive example is the '?line' macro used with common_test (this is
now no longer necessary, but see above about legacy), but I have to
handle for example macros that expand to:
- whole function clauses, in the middle of regular clauses
- guard tests including the 'when' keyword
The parser grammar is basically describing this nice language, except
that any grammar construct or combination thereof may be represented
by a macro call...

In any case, the problems related to smart editors are not the only
ones, there were many people complaining for not so many years ago and
you wrote the oh-so-old-that-Google-can't-find-an-electronic-copy-if-one-even-exists
paper "Delenda est preprocessor" even before that (1998?).

I'm taking the liberty of quoting you here:
"... the preprocessor is violently
at odds with everything else in the language.  One of the papers I
wrote for SERC had the title "Delenda Est Preprocessor".  There
really isn't anything that can be done with the processor that
could not be done better without it.  In particular, one of the
major things about Erlang is the module system combined with hot
loading, but the preprocessor subverts the module system and
causes dependencies between source units that are not and cannot
be tracked by the run time system."
[http://erlang.org/pipermail/erlang-questions/2006-March/019614.html]

best regards,
Vlad