Lexing and parsing, was Re: plain_fsm - for beginners and purists

Ulf Wiger ulf.wiger@REDACTED
Thu Feb 12 09:40:28 CET 2004

On Wed, 11 Feb 2004 10:36:46 -0800 (PST), Thomas Lindgren 
<thomasl_erlang@REDACTED> wrote:

> Interesting idea, but wouldn't the Module have to be
> an entire tokenizer? Just processing a stream of
> tokens (e.g., [... bang, bang, ...] => [..., bangbang,
> ...]) might lead to confusion: how to separate "!!"
> from "! !"?

One easy way to address this would be to have the tokenizer
also account for columns - not just line numbers.

tok_xform([{'!', {L,Col}}, {'!',{L,Col1}}|T]) when Col1==Col+1 ->
    ... % bangbang;

It would be _almost_ backward compatible.  (:


Ulf Wiger, Senior System Architect

This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.

More information about the erlang-questions mailing list