Search code examples
regexkdbintellij-pluginq-langjflex

Regex syntax for support q/k/kdb+ comment using JLexer


Q:

What do I need using for my lexer logic? Only regex or maybe special functions of lexer?

Where does the mistake in my syntax for q multi-comment?

Details:

I'm trying to write intellij idea plugin for k/q/kdb+ (wiki, q/kdb+), and my plugin based on another k language idea plugin.

For beginning I'm trying to write lexer using JLexer (this is standard way for idea plugin). I need support q lang comments. And I have problems for multi-line comments.

My flex file (this syntax works unstable):

LINE_WS=[\ \t\f]
WHITE_SPACE={LINE_WS}+
NEWLINE=\r|\n|\r\n
MULTY_COMMENT={NEWLINE} \/ {WHITE_SPACE}* {NEWLINE} (([^\r\n\\][^\r\n]* {NEWLINE})|{NEWLINE})* \\
// ...
<YYINITIAL> {
   {WHITE_SPACE}                { return com.intellij.psi.TokenType.WHITE_SPACE; }
   {MULTY_COMMENT}              { return COMMENT; }
   // ...

The syntax of q language comments (see more):

Valid comments:

  • / this is comment

  • x: 1; / after '/' we see comment, spaces are important

  • / this is a comment \

  • / after single "/" - all lines are comment if we don't find: NEW_LINE + "\"

Invalid comments:

  • / this is't a comment, break line is important \

  • x: 1;/ this is't a comment, spaces are important

  • \ this is't a comment /

Links:


Solution

  • This code describes q-like multi-line comment for JFlex:

    LINE_WS=[\ \t\f]
    WHITE_SPACE={LINE_WS}+
    NEWLINE=\r|\n|\r\n
    MULTY_COMMENT=\/ {WHITE_SPACE}* {NEWLINE} (([^\r\n\\][^\r\n]* {NEWLINE})|{NEWLINE})* \\
    // ...
    <YYINITIAL> {
       {WHITE_SPACE}                { return com.intellij.psi.TokenType.WHITE_SPACE; }
       ^{MULTY_COMMENT}             { return COMMENT; }
       // ...