I am working on a compiler for cool as compiler course assignment but I'd like to code in cool language in notepad++ so I need to define this language for notepad++, I already wrote a lexer definition in flex format and I'm going to develop a parser for cool. Is there anyway to use these material to define the language for notepad++?
I would think so. (I don't use Notepad++ because I don't use Windows, so I have no good way to validate that theory.)
As I understand it, Noteoad++ is based on Scintilla, and Scintilla provides a mechanism to add custom lexers, using the protocol described in the file doc/Lexer.txt in the Scintilla source.
Basically, to make a flex-generated lexer process text using that protocol, you will either need to read the text to be scanned into a temporary buffer, or show yylex how to read the text (which means compiling your lexer with a custom YY_INPUT macro).
I'd go for the first option, because in order to customize YY_INPUT, you need to make sure that several values are visible in the functions which include expansions of YY_INPUT and, unfortunately, the authors of flex never really took that sort of need into account. (It could be done, though. I'm just avoiding the long description of how to do it.)
In order to provide the correct arguments to styler.ColourTo(), you'll need to track the current scintilla cursor position. To do that, initialise a cursor variable to startPos and increment it with cursor += yyleng in every action. Flex makes it easy to do something in every action: just define the macro YY_USER_ACTION. (NOTE: If you use yyless or yymore, you'll need to correct the cursor position in the corresponding actions.)