[erlang-questions] Erlang AST with locations

[Kenneth Lundin]

On 10/10/07, Robert Virding <rvirding@REDACTED> wrote:
> On 05/10/2007, Joel Reymont <joelr1@REDACTED> wrote:
> > The result of erl_scan:string/1 has line numbers in it but these get
> > lost once erl_parse:parse/1 runs.
> >
> > I would like to have an AST with locations, is there a way to
> > accomplish that?
>
> Sorry I don't really understand what you mean. If you do
>
For example from erl_scan:string below you want the result (in
principle,but maybe
somewhat optimized and compressed)
{ok,[{atom,Location1,abc},
>      {'(',Location2},
>      {'[',Location3},
>      {atom,Location3,a},
>      {']',Location4},
>      {')',Location5},
>      {'->',Location6},
>      {integer,Location7,1},
>      {dot,Location8}],
>     2}

where Location could be {Line,Col,SourceText}
or {FilePos,Length} or {Line,Col,FilePos,OriginalText} to be as
flexible as possible
and for each token. But of course OriginalText is not necessary for all tokens


> 29> erl_parse:parse(Ts).
> {ok,{function,1,
>               abc,
>               1,
>
> [{clause,1,[{cons,1,{atom,Location1,a},{nil,Location2}}],[],[{integer,Location3,1}]}]}}
>
> you that you do have the line numbers in the AST. If you use parse_term,
> however, line numbers are removed as it is the term which returned not the
> AST.
> > On a related note, I think it's a bummer that there's no
> > erl_scan:binary/1.

I don't agree
Why should it be important to have a erl_scan:binary?
When it is so damn easy to do:
{ok,Bin} = file:read_file(File),
Str = binary_to_list(Bin),
erl_scan:string(Str),

Why should we double the same code for a list of characters and a
binary of bytes?
when we easily can do as above.

Of course we can provide a erl_scan:binary/1 which works as above
if that makes enough amount of people happy.

I think we should have a scanner which supports "Locations" in the
original distribution so that Editor and IDE projects as ErlIde don't
need to invent their
own special scanner.
It will happen quite soon but I can't make any promises regarding when.


/Kenneth (Erlang/OTP group at Ericsson)