[erlang-questions] speed of neotoma parser

[French, Michael]

The concept of 'token' is fluid in PEGs. The terminal/non-terminal distinction might not work. For example, the definition of 'alphanumeric' might appear in several different 'tokens', rather than repeating char classes, which means the token rules become non-terminals.

Maybe use a hint in the grammar, like using an upper-case name for rule LHS to indicate a 'token' (which is similar to a convention in Antlr). Then always memo-ize the token rules, but not (necessarily) the others that have lower-case rule names.

BR
Mike

________________________________________
From: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED] on behalf of Joe Armstrong [erlang@REDACTED]
Sent: Friday, November 13, 2015 11:25 PM
To: Sean Cribbs
Cc: Erlang-Questions Questions
Subject: Re: [erlang-questions] speed of neotoma parser

PEG parsers are notoriously inefficient. How about having a separate
tokenization pass, and parse token instead of characters. At a guess
this would be far faster since you'd backtrack over completed tokens
rather than characters.

/Joe

On Fri, Nov 13, 2015 at 3:02 PM, Sean Cribbs <seancribbs@REDACTED> wrote:
> Max,
>
> Do you have a link to your grammar? I can probably poke at it and give you
> some tips.
>
> However, I am well aware of performance problems with neotoma -- with large
> grammars or large inputs it drags. Yes, there are general problems for PEGs
> in Erlang, but its current implementation is particularly naive and
> wasteful. I'm working on a rewrite, but it's a complete overhaul (and more
> faithful to the thesis and reference implementation "Pappy"). Since it's not
> core to my day-job, I've only been able to work on the rewrite occasionally
> in my free time.
>
> On Thu, Nov 12, 2015 at 12:07 PM, Max Lapshin <max.lapshin@REDACTED> wrote:
>>
>> Yes, Louis, I also think that there may be a simple way of speeding it up.
>>
>> I'm only afraid that I will have to open my university book and remember
>> what LL-1 means and how it differs from LALR =)
>>
>> Ok, will try to profile it first.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions