Search code examples
ctoken

Are identifiers separate tokens only in select cases?

1. Lexical Pitfalls:

For another example, consider this statement:

if (x < big) big = x;

Each non-blank character in this statement is a separate token, except for the if keyword and the two instances of the identifier big.

In fact, C programs are broken into tokens twice....

— From C Traps and Pitfalls, Andrew Koenig.

My question is: Why is the if keyword and the two instances of the identifier big not separate tokens? What are they?

Solution

  • The text means to say "every character in this expression is a token, except the characters used to form if and big". In case of if then if is a token, but not the characters i and f respectively.

    That is, this expression consists of the tokens if, (, x, <, big, ), big, =, x, ;.