Generating Custom JavaDoc In Antlr 3

I am implemented a grammar language in Antlr 3 for my company. Currently, when the code is generated there is no comments. I would like to code comments so that the custom comments appear. For example:

/**
* This class does something.
* @author name
*/
public class GrammarLanguageExample() {
}

An example of the grammar language shown below:

grammar Example;

options {
output=AST;
}

@header {
package com.example;
}

@rulecatch {
// ANTLR does not generate its normal rule try/catch
catch(RecognitionException e) {
    throw e;
}
}


insert      : 'INSERT INTO table' 'VALUES';
QUOTE       : '"';
SPECIALCHAR : '-'|'~'|;
STRING  : QUOTE('a'..'z'|'A'..'Z'|SPECIALCHAR|WS)+QUOTE;
WS: (' '|'\t'|'\f'|'\n'|'\r')+ { $channel = HIDDEN;}; // hide token from parser

The above grammar is only an example and is not the full grammar. What happens currently when the code is generated by AntlrWorks 3.4:

I get two Java files: ExampleLexer and ExampleParser.

In the ExampleLexer, there is no package information. I would expect it to go into com.example package whereas the ExampleParser contains the package com.example. Everytime the grammar is changed, I have to type in the com.example for the Lexer code to compile. How can I change the code in the grammar so that the package information is added?

Also, how would I add custom JavaDoc comments? I would like in this format:

package com.example;

Then import statements;

/**
* This class is used for Lexer.
* @author name
*/
public class ExampleLexer() {
}

package com.example;

Then import statements

/**
* This class is used for parsing.
* @author
*/
public class ExampleParser() {
}

I would also like to remove the auto generated comments before the package information, that tells when the grammar was generated. How is this possible?

I hope you can help. I did search on Google but couldn't find anything about this.

Solution

In the ExampleLexer, there is no package information. [...] How can I change the code in the grammar so that the package information is added?

For a combined grammar file (one that produces both a lexer and a token parser), specifying the package names requires specifying two header sections: one for the lexer and one for the parser. As you've discovered, @header applies only to the parser. Here is an example that uses both header sections:

@parser::header {
    package com.example;
    //parser's imports go here.
}

@lexer::header { 
    package com.example;
    //lexer's imports go here.
}

Also, how would I add custom JavaDoc comments?

I think a combined grammar can only specify a Javadoc comment for the parser class, not the lexer. This is done by adding a doc comment before the grammar is declared:

/**
* This class is used for parsing.
* @author name
*/
grammar Example;

The output is what you'd expect:

/**
* This class is used for parsing.
* @author name.
*/
@SuppressWarnings({"all", "warnings", "unchecked"})
public class ExampleParser extends Parser {
 //etc

I think the only way to get a Javadoc comment for the lexer class is to create a grammar file specifically for the lexer and give it a doc comment:

/**
* Lexer!
* @author name
*/
lexer grammar ExampleLexer;

The output is then:

/**
* Lexer!
* @author name
*/
@SuppressWarnings({"all", "warnings", "unchecked"})
public class ExampleLexer extends Lexer {
     //etc

So you'll have to decide whether you want a combined grammar file with Javadoc only on the parser or two grammar files and all the Javadoc you can handle.

I would also like to remove the auto generated comments before the package information, that tells when the grammar was generated. How is this possible?

ANTLR does not provide a command line option to allow this, so it's unlikely that it can be done automatically.

If you expect lots of visitors poking around in your generated files (and it sounds like you do), consider making it clear that the files are in fact generated, and that the reader should look elsewhere for complete documentation (and should not make changes). You can do this easily enough by adding a big, ugly comment at the top of the file, like so:

@parser::header {

    //!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    //!!!                                                            !!!
    //!!! THIS CODE IS AUTOMATICALLY GENERATED! DO NOT MODIFY!       !!!
    //!!! Please refer to file Example.g for grammar documentation.  !!!
    //!!!                                                            !!!
    //!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    package com.example;
}