[erlang-questions] speed of neotoma parser

[Max Lapshin]

I've sent example from production to Sean.

(flussonic@REDACTED)2> timer:tc(fun() ->
config2_format:parse(element(2,file:read_file("big.conf"))), ok end).

{29727926,ok}

On Sat, Nov 14, 2015 at 4:05 PM, French, Michael <michael.french@REDACTED>
wrote:

> The concept of 'token' is fluid in PEGs. The terminal/non-terminal
> distinction might not work. For example, the definition of 'alphanumeric'
> might appear in several different 'tokens', rather than repeating char
> classes, which means the token rules become non-terminals.
>
> Maybe use a hint in the grammar, like using an upper-case name for rule
> LHS to indicate a 'token' (which is similar to a convention in Antlr). Then
> always memo-ize the token rules, but not (necessarily) the others that have
> lower-case rule names.
>
> BR
> Mike
>
> ________________________________________
> From: erlang-questions-bounces@REDACTED [
> erlang-questions-bounces@REDACTED] on behalf of Joe Armstrong [
> erlang@REDACTED]
> Sent: Friday, November 13, 2015 11:25 PM
> To: Sean Cribbs
> Cc: Erlang-Questions Questions
> Subject: Re: [erlang-questions] speed of neotoma parser
>
> PEG parsers are notoriously inefficient. How about having a separate
> tokenization pass, and parse token instead of characters. At a guess
> this would be far faster since you'd backtrack over completed tokens
> rather than characters.
>
> /Joe
>
> On Fri, Nov 13, 2015 at 3:02 PM, Sean Cribbs <seancribbs@REDACTED> wrote:
> > Max,
> >
> > Do you have a link to your grammar? I can probably poke at it and give
> you
> > some tips.
> >
> > However, I am well aware of performance problems with neotoma -- with
> large
> > grammars or large inputs it drags. Yes, there are general problems for
> PEGs
> > in Erlang, but its current implementation is particularly naive and
> > wasteful. I'm working on a rewrite, but it's a complete overhaul (and
> more
> > faithful to the thesis and reference implementation "Pappy"). Since it's
> not
> > core to my day-job, I've only been able to work on the rewrite
> occasionally
> > in my free time.
> >
> > On Thu, Nov 12, 2015 at 12:07 PM, Max Lapshin <max.lapshin@REDACTED>
> wrote:
> >>
> >> Yes, Louis, I also think that there may be a simple way of speeding it
> up.
> >>
> >> I'm only afraid that I will have to open my university book and remember
> >> what LL-1 means and how it differs from LALR =)
> >>
> >> Ok, will try to profile it first.
> >>
> >> _______________________________________________
> >> erlang-questions mailing list
> >> erlang-questions@REDACTED
> >> http://erlang.org/mailman/listinfo/erlang-questions
> >>
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151114/17aaaba7/attachment.htm>