[erlang-patches] Scan "=<<" as something useful

Raimo Niskanen <>
Mon Apr 29 15:05:45 CEST 2013


On Mon, Apr 29, 2013 at 11:41:35AM +0200, Anthony Ramine wrote:
> Hello Raimo,
> 
> If I read the documentation, I don't see anywhere where it is written that erl_scan is supposed to be greedy. Nor I see any syntax knowledge if we just say that erl_scan will avoid returning two consecutive comparison operators if it can. That is just knowledge that '=<' '<' makes no sense, and I can't think of any language where it does.

But that knowledge is aquired by doing an infinite look-ahead
for this operator, and knowing that =< and < are both comparision
operators _is_ syntax knowledge as are all other reasons for
knowing that =< and < are not allowed in sequence.

> 
> Erlang is and has always been pragmatic, letting such an embarrassing syntax wart for the sake of beauty isn't what should be done here, in my opinion. Pragmatism is what allows us to have a nice syntax for specs and callbacks; which wouldn't be possible without an ugly hack too[1].
> 
> What would break if that patch is accepted? The answer is nothing.

Here are the old arguments:
    http://erlang.org/pipermail/erlang-questions/2003-May/008751.html

It heats up at:
    http://erlang.org/pipermail/erlang-questions/2003-May/008923.html

...and the subject pops up occasionally:
    http://erlang.org/pipermail/erlang-questions/2010-July/052587.html

One final straw back then was this: the typo "A=<<<1>>" accidentally
becomes valid syntax with the proposed change.

> 
> Regards,
> 
> [1] https://github.com/nox/otp/blob/master/lib/stdlib/src/erl_parse.yrl#L543-552
> 
> -- 
> Anthony Ramine
> 
> Le 29 avr. 2013 à 11:18, Raimo Niskanen a écrit :
> 
> > On Mon, Apr 29, 2013 at 10:53:42AM +0200, Raimo Niskanen wrote:
> >> On Sun, Apr 21, 2013 at 10:15:28PM +0200, Anthony Ramine wrote:
> >>> Hello,
> >>> 
> >>> I've noticed that Erlang/OTP own code tend to be space-free, e.g. {a,
> >>> b} is rather written as {a,b}. Then there is the occasional Bin=
> >>> <<42>> with a space because erl_scan will scans "=<<" as '=<','<'
> >>> instead of '=','<<'. This patch teaches erl_scan how to not do that.
> >>> 
> >>> There is no backwards compatibility problem as '=<','<' can't appear
> >>> in valid code.
> >> 
> >> When I tried to fix this years ago while rewriting the scanner just after
> >> the binary syntax was introduced I got lecutured (I think by Robert)
> >> that it was a bad change because it introduces syntax knowledge
> >> into the scanner.
> >> 
> >> The scanner is today greedy so it collects as many characters as it
> >> can into a valid token, and "=<" is a valid token.  When it is
> >> followed by "<" that becomes a syntax error in the parser.
> >> 
> >> I kind of miss this argument this time around...
> >> 
> > 
> > If I read your patch correctly you also fix this variant:
> > 
> >    B=<<<<1>>/binary,2>>.
> > 
> > which means the scanner reads an arbitrary number of "<" after
> > "=<" and then based on if they are odd or even decides to be
> > greedy or not, and that is really much and ugly syntax knowledge
> > for a scanner...
> > 
> > I got a big NO for this change.
> > 
> >>> 
> >>>        git fetch https://github.com/nox/otp.git disambiguate-equal-binary
> >>> 
> >>>        https://github.com/nox/otp/compare/disambiguate-equal-binary
> >>>        https://github.com/nox/otp/compare/disambiguate-equal-binary.patch
> >>> 
> >>> Regards,
> >>> 
> >>> --
> >>> Anthony Ramine
> >>> _______________________________________________
> >>> erlang-patches mailing list
> >>> 
> >>> http://erlang.org/mailman/listinfo/erlang-patches
> >> 
> >> -- 
> >> 
> >> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> >> _______________________________________________
> >> erlang-patches mailing list
> >> 
> >> http://erlang.org/mailman/listinfo/erlang-patches
> > 
> > -- 
> > 
> > / Raimo Niskanen, Erlang/OTP, Ericsson AB
> > _______________________________________________
> > erlang-patches mailing list
> > 
> > http://erlang.org/mailman/listinfo/erlang-patches
> 

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


More information about the erlang-patches mailing list