[erlang-patches] Return end locations in erl_scan

[Anthony Ramine]

I created the 'file' attribute because erl_lint and its friends
used to zip the file and line number together in a tuple, breaking
the assumption that if the location is a tuple, it is a pair of
line and column numbers, that also doest not respect the AST
documentation from absform.xml. The two first commits are also
in the compile-column-numbers branch (the 'file' thing and a
bugfix).

I should have made it clear in the initial mail, but I did write
that in my commit messages :)

Regarding diagnostics and macro tracking more generally, this should
be done in step: first, a token/AST node range (its begin and end
locations) coming from a macro expansion should reflect what is written
in the source file. For example, if I write in foo.erl ?MODULE at line
1 and column 10, I want the resulting atom 'foo' to begin at line 1 and
column 10 and end at line 1 and column 17, not column 13; second,
for non-builtin macros (e.g. not ?MODULE), a new attribute 'from' should
be created, of type {'from',[{File::string(),Start::location(),End::location()}]}.
We don't actually need special tokens.

In the end, that is not related to the topic at hand, which is
to teach erl_scan to return end locations.

The one thing we should discuss right now is whether this patch should
be included separately right now, or should I come back when I've
finished teaching erl_parse to return locations range, epp to track macro
expansions and compile to actually use all these informations to spit
out related source code in the shell when compilation fails.

Regards,

-- 
Anthony Ramine

Le 21 mars 2013 à 12:20, Vlad Dumitrescu a écrit :

> 
> On Thu, Mar 21, 2013 at 10:33 AM, Anthony Ramine <n.oxyde@REDACTED> wrote:
> Maybe, but still if we want diagnostics we need to properly track
> macro expansions.
> 
> See, it actually helps checking the code :-) Why didn't you mention that there is a new token attribute, "file"? Suddenly all makes more sense :-)
> 
> But there is something missing (or maybe it is in the pipeline) -- the really useful part would be if the tokens produced by expanding a macro would point to the file defining it, right? 
> 
> On the other hand, I would sometimes want to have the tokens for the non-preprocessed source (for use by the editor) and it would be extra nice if these could be produced by a single scanning. Maybe we could have special tokens for that, that are treated by other tools as whitespace? In this case, the 'end' position is crucial to have because the special macro tokens will overlap with their expansion.
> 
> Maybe this is a discussion better taken in a separate thread, on erlang-questions?
> 
> regards,
> /Vlad