Search code examples
pythonpython-3.xpython-3.9pegpython-3.10

What is the purpose of bitwise_or in Python PEG?


What does bitwise_or mean in PEG grammar? More accurately, there are lots of bitwise_or in contexts where it's not even parsing for | yet there are occurrences. Does bitwise_or serve any other purpose in PEG other than being the | in Python?

Example extracted from Python PEG:-

comparison[expr_ty]:
    | a=bitwise_or b=compare_op_bitwise_or_pair+ {
        _PyAST_Compare(
            a,
            CHECK(asdl_int_seq*, _PyPegen_get_cmpops(p, b)),
            CHECK(asdl_expr_seq*, _PyPegen_get_exprs(p, b)),
            EXTRA) }
    | bitwise_or

Note the word bitwise_or here. The question is about that not the vertical bar in PEG.


Solution

  • The "bitwise or operator" aka | has the lowest precedence of regular binary operators. The only binary operators with lower precedence are comparison operators, which are subject to chaining – for example, a < b < c is roughly equivalent to a < b and b < c – and thus behave specially.

    For a PEG parser, precedence is usually encoded using precedence climbing. That means a lower precedence clause matches either itself or the next precedence clause. As such, the operator precedence "| < ^ < & < ..." is encoded in a ladder of bitwise or, bitwise xor, bitwise and, and so on:

    bitwise_or:
        | bitwise_or '|' bitwise_xor 
        | bitwise_xor
    bitwise_xor:
        | bitwise_xor '^' bitwise_and 
        | bitwise_and
    bitwise_and:
        | bitwise_and '&' shift_expr 
        | shift_expr
    

    This makes bitwise_or the "entry point" that matches all binary operators: it may defer to bitwise_xor, which may defer to bitwise_and, and so on to the highest-precedence operator. Notably, it means the grammar rule bitwise_or can match input that does not contain the operation "bitwise or" – for example, bitwise_or matches a ^ b.

    Thus, bitwise_or is used in any position a binary operator may occur.