pypeg cannot compose grammar with list?

Question

class A(List):
    grammar = [(Symbol, ':', Symbol), Symbol]

compose(parse('a', A))

This raises a compose error while parsing is fine.

I can only bypass it by using a dummy class:

class B(List):
    grammar = Symbol, ':', Symbol
class A2(List):
    grammar = [B, Symbol]

compose(parse('a', A2))

Now it composes correctly.

What is wrong here?

Answer 1

You found a bug in pyPEG.

When composing List objects, pyPEG uses a stack to keep track of which list item should be composed next. Tuples in the grammar remove items from the stack, but never put them back, even if the tuple fails to match. This causes an IndexError when the code tries to access an element in the empty stack.

For your example, it looks something like this:

    Stack                    Grammar                      Action                
-----------------------------------------------------------------------
[Symbol('a')]    [(Symbol, ':', Symbol), Symbol]    matched, pop stack
                   ^^^^^^

[]               [(Symbol, ':', Symbol), Symbol]    str, append to text
                           ^^^

[]               [(Symbol, ':', Symbol), Symbol]    no match
                                ^^^^^^

[]               [(Symbol, ':', Symbol), Symbol]    IndexError!
                                         ^^^^^^

I've submitted a pull request with a fix.