lex generator

[Thomas Lindgren]

I just added a lexical analyser generator to jungerl
as
lib/smart_exceptions/src/lex.erl (sorry about the
unobvious location).

Usage:

1. Generate a lexing table from a collection of regexp
rules. (Regexps compatible with the regexp module; see
comments and examples in lex.erl for more info.)

  lex:regexps_to_table([Regexp_rule]) -> Table

Generating a lexer from rules is fast, e.g. 55 ms on
my AMD 1300+ for the lex:erlang_lex/0 specification
found in lex.erl. You can of course generate and use
any number of lex tables if you wish.

2. Use the table to lex a string into tokens

  lex:longest(Table, String) -> [Token]

longest/2 is the provided driver function. I haven't
measured its speed, but it seems reasonable for
interactive use. You can use longest/2 as a template
for writing other lex-driver functions.

A somewhat realistic example lexer inspired by
erl_scan is provided (lex:real_erlang_lex/0), along
with some other examples/tests.

Best,
Thomas



	
		
__________________________________
Do you Yahoo!?
Win a $20,000 Career Makeover at Yahoo! HotJobs  
http://hotjobs.sweepstakes.yahoo.com/careermakeover