Fix for A=<<1>>

[Robert Virding]

The problems are at different levels:

1. There is a fundamental difference in scanning atoms/strings and =<<<<<.
You have basically introduced infinite look-ahead (at least to the end of
the file) to try and determine what the first token actually is, for
everything else there is one character look-ahead. (As is the grammar which
is one LALR1)

2. There are at least three different "one key-press typos" which can give
"A=<<1>>":

A==<<1>>
A=<<<1>>
A= <<1>>

Two tests and a match. How can you so so certain what I meant as to choose
one? Introducing a match when I really meant a test will introduce a
fundamental semantic change to the code, both as to return value and to the
error case. Also if I had mistyped the variable name then this *really*
changes the meaning of the code. And it does it automagically, completely
silently and in a way which can make it extremely difficult for the
programmer to find!

3. Nothing should try to correct its input, especially not something as
fundamental as the scanner. Especially when the "correction" is not
unambiguous. Just because you now don't think the chances of meaning
something else is slim doesn't mean other people think the same way. Or that
you might not think so in the future. :-)

4. I personally always (or almost always) use spaces to separate my symbols
so I don't really see what the problem is.

My basic premise is that you can not add an "improver" which works silently
and is only sometimes correct. Will you take the responsibility when this
generates an serious, invisible error in someones code?

The trouble with writing code at this level is that you always have to try
and handle the case when users do things which you had assumed was so stupid
or strange,even though it is legal,  that no one would do it, in this case
mean "A=<<<1>>" when they  wrote "A=<<1>>". I remember that a relatively
early version of the JAM compiler could not handle the case when you did a
send as an argument to a function to get the message in as an argument,
(foo(X ! <big evaluation>, ...)). The premise was that send is used for side
effects not return values. Of course someone did just this and it crashed.

Robert

P.S. Liked the new code. You need to fix the comments. Can't the fun be
generated each suspend and include the state? I think it would make things
easier.