erl_scan:string ignores /' and /"

[Joe Armstrong]

I think things are as they should be.

writing\C is just the same as writing C if C is not a one of the special
formatting characters s,n,r,... ^ or an octal digit

Thus
 
\' is just the same as '
\" is just the same as "

thus erl_scan:string("\''") is the same as
     erl_scan:string("'")

which should give an error since ' is NOT a valid token, it is the
start of an atom.

try say,

	> erl_scan:string("\'a\'").
        {ok,[{atom,1,a}],1}

Queries 25 and 26 seem correct since
erl_scan:string expects a string containing completed tokens.

For incomplete stings you can call the re-entrant
scanner erl_scan:tokens, like this:

1> erl_scan:tokens("", "\'", 1).
{more,{[{'\'',1}],"'",1,1}}
2> {more, M} = v(-1).
{more,{[{'\'',1}],"'",1,1}}
3> erl_scan:tokens(M, "abc'", 1).
{more,{[],"'cba'",1,1}}
4> {more,M1}=v(-1).
{more,{[],"'cba'",1,1}}
5> erl_scan:tokens(M1, ",123. ", 1).
{more,{[{'\'',1}],"'.321,'cba'",1,1}}
6> erl_scan:tokens(M1, ",123. ", 1).
{done,{ok,[{atom,1,abc},{',',1},{integer,1,123},{dot,1}],1},[]}


Hope that helps

/Joe


On Sun, 17 Aug 2003, Roger Price wrote:

> "Concurrent Programming in Erlang", chapter 2.1.2 says "Within a quoted
> atom the following conventions apply"...  Chapter 2.1.4 says "Within a
> string the quoting conventions used within an atom also apply."  However
> erl_scan:string/1 seems to ignore \' and \".  Here are the results for all
> the conventions:
> 
> 15> erl_scan:string("\b") .
> {ok,[],1}
> 16> erl_scan:string("\d") .
> {ok,[{'\d',1}],1}
> 17> erl_scan:string("\e") .
> {ok,[],1}
> 18> erl_scan:string("\f") .
> {ok,[],1}
> 19> erl_scan:string("\n") .
> {ok,[],2}
> 20> erl_scan:string("\r") .
> {ok,[],1}
> 21> erl_scan:string("\t") .
> {ok,[],1}
> 22> erl_scan:string("\v") .
> {ok,[],1}
> 23> erl_scan:string("\\") .
> {ok,[{'\\',1}],1}
> 24> erl_scan:string("\^A") .
> {ok,[],1}
> 25> erl_scan:string("\'") .
> {error,{1,erl_scan,{string,39,[]}},1}
> 26> erl_scan:string("\"") .
> {error,{1,erl_scan,{string,34,[]}},1}
> 27> erl_scan:string("\266") .
> {ok,[{'¶',1}],1}
> 28> erl_scan:string("\266\
> 28> \266") .
> {ok,[{'¶',1},{'¶',2}],2}
> 
> Roger
> 
> 
erl