Alex requies the result token type to be some function that takes a AlexPosn

Question

I'm trying to write a lexer that lexes c-style comments and nothing else(for now).

{
module Lexer where

import Prelude hiding (head, take, tail)
import Data.ByteString.Lazy
}

%wrapper "monad-bytestring"

@not_bc_end = ~\* | \* ~\/


tl :-
  <0>   $white+             ;
  <0>   "/*"                { tok (\p s -> BCBegin p) `andBegin` bc }
  <bc>  .+ / not_bc_end     { tok (\p s -> BCContent p s) }
  <bc>  "*/"                { tok (\p s -> BCEnd p) `andBegin` 0 }
  <0>   "//"                { tok (\p s -> LCBegin p) `andBegin` lc }
  <lc>  .*$                 { tok (\p s -> LCContent p s) }

{
tok :: (AlexPosn -> ByteString -> Token) -> AlexInput -> Int64 -> Alex Token
tok f (p, _, bs, _) len = pure $ f p (take len bs)

data Token
  = LCBegin AlexPosn
  | LCContent AlexPosn ByteString
  | BCBegin AlexPosn
  | BCEnd AlexPosn
  | BCContent AlexPosn ByteString
  | End AlexPosn

alexEOF = pure End
}

The codegen succeeded but the compilation failed with the following error:

templates/wrappers.hs:288:9: error:
    • Couldn't match type ‘Token’ with ‘AlexPosn -> Token’
      Expected type: Alex (AlexPosn -> Token)
        Actual type: Alex Token
    • In a stmt of a 'do' block: action (ignorePendingBytes inp__) len
      In the expression:
        do alexSetInput inp__'
           action (ignorePendingBytes inp__) len
      In the expression:
        let len = n' - n
        in
          do alexSetInput inp__'
             action (ignorePendingBytes inp__) len

The compiler complains that the token type (wrapped in Alex) have to take an argument of the type AlexPosn. I got confused because there's already an AlexPosn in the AlexInput and the user guide explicitly stated that the token type can bu anything as long as they are kept the same across all the tokens.

Changing the definition of tok to the following solved the problem:

tok :: (AlexPosn -> ByteString -> Token) -> AlexInput -> Int64 -> Alex (AlexPron -> Token)
tok f (_, _, bs, _) len = pure $ \p -> f p (take len bs)

But I have no idea why is Alex focing the token to take an AlexPosn

Answer 1

I accidentally added a field of type AlexPosn to the constructor End