I have a problem with Parsec recognizing comments when parsing mustache templates.
The various mustache tags all start with {{ include the block comment ({{!comment}}).
I have set commentStart and commentEnd to {{! and }} in my TokenParser.
Whenever I add comments to a template, Parsec complains that the comment is unexpected.
It expects a mustache variable instead, since that is the only token that matches {{.
When does Parsec remove comments? I thought it would happen before the source hits my parser?
Parsec doesn't remove comments. In a TokenParser, comments are subsumed under white space, so
whiteSpace tokenParser
skips comments and ordinary white space (blanks, tabs, newlines, ...).
Usually, you use lexeme parser to skip all white space following a lexeme, then you only need one initial white-space-skipping for the top-level parser to skip any leading white space in the source, afterward, all white space (including comments) is handled automatically (by the TokenParser that makeTokenParser creates).
If you don't use lexeme and handle white space manually, you must take care of tokens/lexemes that are a prefix of the comment delimiter. If you try the prefix first, that will succeed, but only consume part of the comment delimiter, in this case leaving the '!' for the variable parser, which then fails.