I'm reading the Python documentation 2.76
This is the lexical syntax of the 3 kinds of boolean operations :
or_test ::= and_test | or_test "or" and_test
and_test ::= not_test | and_test "and" not_test
not_test ::= comparison | "not" not_test
This is the lexical syntax for comparision:
comparison ::= or_expr ( comp_operator or_expr )*
comp_operator ::= "<" | ">" | "==" | ">=" | "<=" | "<>" | "!="
| "is" ["not"] | ["not"] "in"
and the lexical syntax for or_expr:
and_expr ::= shift_expr | and_expr "&" shift_expr
xor_expr ::= and_expr | xor_expr "^" and_expr
or_expr ::= xor_expr | or_expr "|" xor_expr
The Notation in the document is explained here:
Now comes the question:
Is not_test ::= comparison | "not" not_test parsed like
not_test ::= comparison | ("not" not_test)?
If 1. is true, a valid comparison is also a valid not_test.(e.g. 1 < 2 is a not_test even without a not in it.)
In addition, because a valid and_test can just be a single valid not_test, 1 < 2 is a valid and_test as well.
The same goes for the or_test.
Then what is a valid comparison? 1 < 2 fits the pattern obviously. And thoese bitwise comparison expr are aslo valid comparison. What's in common is that the presense of at least one operator('>','<',or bitwise stuff) is required. (I'm not sure.)
Here comes to the weird part. Consider x and y for example. According to
and_test ::= not_test | and_test "and" not_test
and_test ::= not_test | (and_test "and" not_test) # parsed like this I believe?
If it is true that x and y is a valid and_test (it can never be a not_test for the presence of and), then x must be a valid and_test which can just be one valid not_test. And y must be a valid not_test too.
A not_test can either be a single comparison or another not_test preceeded by not. So a not_test is basically zero or more nots followed by one comparison. What matters now is the lexical syntax of comparison.
According to 4., a comparison must have at least one operator whatever. But this is conflict with the following example:
assign x = 3, y = 4 . 3 and 4 seems a valid and_test.
But I don't see how 3 or 4 can be valid comparison.
Where did I go wrong?
The formal grammar specifies precedence in addition to simply defining the parts of the language. That's why many of the grammar nodes for expression parts contain a left-hand alternative which doesn't fit the actual name of the node; these alternatives specify a grammar node which has the next-highest precedence (thus, when expanding the tree, you would expand low precedence operations followed by high precedence operations).
This is clearest in the arithmetic operators:
power ::= primary ["**" u_expr]
u_expr ::= power | "-" u_expr | "+" u_expr | "~" u_expr
m_expr ::= u_expr | m_expr "*" u_expr | m_expr "//" u_expr | m_expr "/" u_expr
| m_expr "%" u_expr
a_expr ::= m_expr | a_expr "+" m_expr | a_expr "-" m_expr
shift_expr ::= a_expr | shift_expr ( "<<" | ">>" ) a_expr
Observe that u_expr has power as one of its alternatives, since power binds tighter than unaries (at least on the left). m_expr can expand to a u_expr because unaries bind tighter than multiplication operations, and a_expr can expand to a m_expr because multiplication operations bind tighter than addition operations. These statements are not attempting to imply that you should consider the statement x * y to actually be an addition operation (though it can be considered a term in a one-term addition, in the grammatical sense).
What this means, basically, is that you can trace along the LHS of the nodes until you reach one that parses your expression.
For example, the expression 3 + 4 is a valid comparison according to the grammar (though it isn't a "comparison" in the general language sense). This is because you can expand it according to the chain
comparison > or_expr > xor_expr > and_expr > shift_expr > a_expr > a_expr + m_expr
and further with
a_expr > m_expr > u_expr > power > primary > atom > literal > integer > decimalinteger > etc...
At each stage, you basically ascend the precedence chain by one rank, which gives a very clean way to determine what the order of operations is.