Search code examples
javaabstract-syntax-treeeclipse-cdt

CDT IASTNode getRawSignature after preprocessor


In CDT core plugin, there is a method getRawSignature of interface IASTNode that was described as:

Returns the raw signature of the IASTNode before it is processed by the preprocessor.
Example:
#define ONE 1
int x=ONE; // getRawSignature() for this declaration would return "int x=ONE;"

But I want to get the String signature after the node is processed by the preprocessor
In the above example, the expected string is: int x=1
How to get this string? I have looked at some other methods, but no one can.


Solution

  • Great question!

    The first thing to note is that, at no point during CDT's processing does code exist in the form of a preprocessed string.

    The processing flow is this:

       Unpreprocessed string
    -> [Lexer]
    -> Unpreprocessed token stream
    -> [Preprocessor]
    -> Preprocessed token stream
    -> [Parser]
    -> Abstract syntax tree
    

    Notice that in preprocessed form, the code only exists as a token stream, not as a string.

    That said, if you had the preprocessed token stream, you could probably use it to construct a preprocessed string.

    Unfortunately, I'm not aware of an easy way to obtain preprocessed tokens, and this mailing list thread suggests there may not be one.

    The closest I think we can come is to re-preprocess the file, and thereby obtain preprocessed token stream for the entire file. This can be done by invoking AbstractCLikeLanguage.createScanner() to obtain an IScanner (it's a protected method, so you'd need to derive from GCCLanguage or GPPLanguage to access it), and calling IScanner.nextToken() repeatedly to obtain the preprocessed tokens.

    That still doesn't quite give you what you want, since you want preprocessed tokens corresponding to a particular AST node. I believe you can compute this by comparing the offset and length of the preprocessed tokens (obtained using IToken.getOffset() and IToken.getLength()) to the offset and length of the AST node (obtained using ASTNode.getOffset() and ASTNode.getLength()), which I believe are in the same numbering space.