[erlang-questions] Debugging yecc grammars: what techniques are available?

[Hugo Mills]

   Thanks to all who replied on this thread, and apologies for not
getting back sooner -- more urgent things got in the way.

On Sun, Feb 05, 2017 at 11:18:39AM +0100, Mikael Pettersson wrote:
> Hugo Mills writes:
>  >    I'm in the process of writing a domain-specific language, using a
>  > custom lexer and a yecc parser. In the process of attempting to add a
>  > new feature to the grammar, I've managed to break what appears to me
>  > to be a completely unrelated piece of the grammar: I'm adding a
>  > structure similar to a list comprehension, and suddenly the integer
>  > comparison operators have stopped working in one group of cases.
>  > 
>  >    What approaches are there for debugging this kind of problem? The
>  > best I can some up with is inspection of the .yrl source in detail,
>  > but that's not helping. How else can I work out what the issue is?
>  > 
>  >    Obviously, I could post the grammar here for inspection by more
>  > experienced minds than mine, but I'd prefer to find better tooling and
>  > techniques rather than mere appeal to higher authority...
> 
> Caveat: while I have experience with various parsing techniques, including
> LALR(1), I have none with the yecc tool.
> 
> Does yecc issue any warnings about shift/reduce or reduce/reduce conflicts?
> If it does, you need to investigate those and resolve them.  With yacc and
> bison it's possible to manually inspect the LALR(1) automaton for debugging.

   This turned out to be my main problem: I actually had 20 unresolved
shift/reduce conflicts, but in the yecc output, they were listed at
the top of a giant list of properly resolved conflicts (using
precedence rules), and I hadn't seen them.

   I fixed up those problems, and several more that arose elsewhere in
the grammar once those had been removed (for example, duplicate
definitions of some structures like comma-separated lists). This fixed
up a bunch of my parser test failures. Then I found that the
problematic integer comparison operators had actually been only
coincidentally passing the unit tests, and that the test failures I
was seeing were correct -- so I fixed a bug in the evaluator code for
those.

   Finally, I worked out how to give function invocation a high
precedence, which has, I think, fixed the last remaining issue I had
with the grammar. (I can now write f(3)(6) and 3+g(3) and get the
correct parse trees).

   I've still got some known bugs in the evaluator, but they're not
parser problems, and I know how to fix those easily enough.

   Thanks,
   Hugo.

-- 
Hugo Mills             | "There's more than one way to do it" is not a
hugo@REDACTED carfax.org.uk | commandment. It is a dire warning.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170217/254262c9/attachment.bin>