[erlang-questions] speed of neotoma parser

[French, Michael]

PEG packrat parser generators in Erlang have a lot of scope for performance problems:
 - text character input, should use binary for large input
 - random access in lists, should use tuples for small input
 - non-memoized performance problem, multiple execution of same parse
 - memoized memory problem, possibly multiple copies of input
 - slow random character access in UTF8 binary, needs btree index?

You could also try:

     https://github.com/joearms/elib1/tree/master/unsupported/epeg

but it uses a different PEG grammar, different embedded Erlang rule binding, and it's 'unsupported'.

Does anyone have any performance comparison of epeg and neotoma?

BR
Mike


________________________________________
From: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED] on behalf of Dmitry Kolesnikov [dmkolesnikov@REDACTED]
Sent: Wednesday, November 11, 2015 11:56 PM
To: Max Lapshin
Cc: Erlang-Questions Questions
Subject: Re: [erlang-questions] speed of neotoma parser

Hello,

I would suggest you to profile it with
https://github.com/virtan/eep

This is very good to tool to catch this type of things

Best Regards,
Dmitry

> On 11 Nov 2015, at 22:48, Max Lapshin <max.lapshin@REDACTED> wrote:
>
> Our Flussonic is using neotoma to parse our own config.
>
> Neotoma creates parser, this parser reads text config.
>
> It was great, users were really happy when we changed from erlang syntax to our own syntax, but now time to pay came.
>
> One user came with 280 KB config file (written by hands, yes). Neotoma parser parses it during 45 seconds.
>
> We've met with this problem earlier in our pulsedb library, when we changed very small (one screen) parser from neotoma to yecc and got 50-100 times speedup.
>
> Now it is not easy to do it, because our config parser takes about 1000 lines of peg file.
>
> I don't undertand what is the problem with speed of neotoma parser:
> 1) is it a nature of state machines that it generate?
> 2) is it a forgotten switch [{disable_slow,true}] in options?
> or is it something else?
>
> Maybe there is a simple thing that we can do with neotoma for it to generate faster parser?
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions