[erlang-patches] Scan "=<<" as something useful

[Kostis Sagonas]

On 04/29/2013 11:41 AM, Anthony Ramine wrote:
> Hello Raimo,
>
> If I read the documentation, I don't see anywhere where it is written that erl_scan is supposed to be greedy. Nor I see any syntax knowledge if we just say that erl_scan will avoid returning two consecutive comparison operators if it can. That is just knowledge that '=<''<' makes no sense, and I can't think of any language where it does.
>
> Erlang is and has always been pragmatic, letting such an embarrassing syntax wart for the sake of beauty isn't what should be done here, in my opinion. Pragmatism is what allows us to have a nice syntax for specs and callbacks; which wouldn't be possible without an ugly hack too[1].
>
> What would break if that patch is accepted? The answer is nothing.

I do not have any strong opinion on the issue under discussion but as a 
bystander I can only state that I see some validity in the arguments of 
both sides:

  - In the "pragmatic" argument/stand taken by Anthony and in all the 
support from poor souls that have been bitten by this language feature.

  - In the technical argument that says that lexical analyzers implement 
what's called "maximal munch"[1] when scanning the input from left to 
right and it would be weird to change and special-case the case where = 
is immediately followed by < depending on (arbitrary) lookahead.

(Aside: on the other hand I find the "this has never bitten me; always 
put a space there" response, not particularly constructive...)


But I also have something technical to add to the discussion:

If the patch is to be accepted and some machinery is to be added to the 
scanner to intelligently break occurrences of =<<, =<<<<, ... into 
tokens, the same cleverness needs to be introduced to the cases of <<<, 
<<<<<, ... which currently also result in syntax errors:

Eshell V5.10.2  (abort with ^G)
1> 42=<<<42>>.
true
2> 42<<<42>>.
* 1: syntax error before: '<<'
2> 42< <<42>>.
true
3> 42<<<<<42>>/binary>>.
* 1: syntax error before: '<<'
3> 42< <<<<42>>/binary>>.
true

The above is also a side-effect of maximal munch from left to right, 
which Anthony's current patch does not handle, if I am not mistaken. 
Otherwise, it would be inconsistent.

Kostis

[1] http://en.wikipedia.org/wiki/Maximal_munch