Fix for A=<<1>>

[Raimo Niskanen]

OK, your argument about the typo "A=<<<1>>" when meaning "A=<<1>>" i.e 
"A = << 1 >>" accidentally becoming a valid syntax "A =< << 1 >>" has 
convinced us to remove this change from erl_scan (at least for R9C).

The debate will probably respawn (reflame) after the R9C release. It is 
easier to add later than to remove. The final word is certainly not said.

/ Raimo Niskanen, Erlang/OTP, Ericsson AB



Robert Virding wrote:
> The problems are at different levels:
> 
> 1. There is a fundamental difference in scanning atoms/strings and =<<<<<.
> You have basically introduced infinite look-ahead (at least to the end of
> the file) to try and determine what the first token actually is, for
> everything else there is one character look-ahead. (As is the grammar which
> is one LALR1)
> 
> 2. There are at least three different "one key-press typos" which can give
> "A=<<1>>":
> 
> A==<<1>>
> A=<<<1>>
> A= <<1>>
> 
> Two tests and a match. How can you so so certain what I meant as to choose
> one? Introducing a match when I really meant a test will introduce a
> fundamental semantic change to the code, both as to return value and to the
> error case. Also if I had mistyped the variable name then this *really*
> changes the meaning of the code. And it does it automagically, completely
> silently and in a way which can make it extremely difficult for the
> programmer to find!
> 
> 3. Nothing should try to correct its input, especially not something as
> fundamental as the scanner. Especially when the "correction" is not
> unambiguous. Just because you now don't think the chances of meaning
> something else is slim doesn't mean other people think the same way. Or that
> you might not think so in the future. :-)
> 
> 4. I personally always (or almost always) use spaces to separate my symbols
> so I don't really see what the problem is.
> 
> My basic premise is that you can not add an "improver" which works silently
> and is only sometimes correct. Will you take the responsibility when this
> generates an serious, invisible error in someones code?
> 
> The trouble with writing code at this level is that you always have to try
> and handle the case when users do things which you had assumed was so stupid
> or strange,even though it is legal,  that no one would do it, in this case
> mean "A=<<<1>>" when they  wrote "A=<<1>>". I remember that a relatively
> early version of the JAM compiler could not handle the case when you did a
> send as an argument to a function to get the message in as an argument,
> (foo(X ! <big evaluation>, ...)). The premise was that send is used for side
> effects not return values. Of course someone did just this and it crashed.
> 
> Robert
> 
> P.S. Liked the new code. You need to fix the comments. Can't the fun be
> generated each suspend and include the state? I think it would make things
> easier.
> 
>