[erlang-questions] leex and case-insensitive match

[Knut Nesheim]

At the moment I'm using the first solution you propose (and proposed
by Joel in the old post). It works out fine. It's cumbersome to write
and the lexer file becomes huge in my case. The generated module is 8k
lines of code. Otherwise, I don't have any problems.

What I would like to do in a perfect world is to use the "i" flag
found in some regex implementations:

    /select/i     : {token, {select, TokenLine, TokenChars}}

Regards
Knut

On Thu, Jun 4, 2015 at 8:15 PM, Jesper Louis Andersen
<jesper.louis.andersen@REDACTED> wrote:
>
> On Thu, Jun 4, 2015 at 7:06 PM, Knut Nesheim <knutin@REDACTED> wrote:
>>
>> For my use case "select", "SELECT", "SeLeCt" means exactly the
>> same.
>
>
> I'm not sure I understand this:
>
> [jlouis@REDACTED ~]$ cat z.xrl
> %% Test
>
> Definitions.
>
> Rules.
>
> [sS][eE][lL][eE][cC][tT] : {token, select}.
>
> Erlang code.
>
> %% Nothing
>
> in an erl shell:
>
> 4> leex:file("z").
> {ok,"./z.erl"}
> 6> c("z.erl").
> {ok,z}
> 9> [z:string(Str) || Str <- ["select", "SELECT", "SeLeCt"]].
> [{ok,[select],1},{ok,[select],1},{ok,[select],1}]
>
> Note how the token parses as the same thing.
>
> [sS][eE][lL][eE][cC][tT] : {token, {identifier,
> string:to_lower(TokenChars)}}.
>
> should also work, if select is not a keyword, but you want to handle it as
> an identifier.
>
> some Regex engines provides convenience notation for [sS], [eE] etc, but it
> tends to be rare enough for keywords that you skip it. Another path through
> the game is the following age-old lexer hack:
>
> [jlouis@REDACTED ~]$ cat y.xrl
> %% Test
>
> Definitions.
>
> ALPHA = [a-zA-Z]
>
> Rules.
>
> {ALPHA}* : {token, analyze_alpha(string:to_lower(TokenChars))}.
>
> Erlang code.
>
> %% Match keywords, and if no keyword matches, regard the token as an
> identifier
> analyze_alpha("select") -> select;
> analyze_alpha(Otherwise) -> {identifier, Otherwise}.
>
> 13> leex:file("y").
> {ok,"./y.erl"}
> 14> c("y.erl").
> {ok,y}
> 15> [y:string(Str) || Str <- ["select", "SELECT", "SeLeCt"]].
> [{ok,[select],1},{ok,[select],1},{ok,[select],1}]
>
> Do any of these work for you?
>
> --
> J.