Using lexeme in parsec

Question

I don't understand how to use the lexeme function

I have seen the above question, but I still don't understand.

The example in the documentation, for instance, also does not work.

mainParser  = do{ whiteSpace
                ; ds <- many (lexeme digit)
                ; eof
                ; return (sum ds)
                }

Answer 1

Disclaimer: I haven't used Parsec yet. That being said, lexeme is a field of GenTokenParser s u m. If you inspect it's type in GHCi, you'll end up with

lexeme :: GenTokenParser s u m -> ParsecT s u m a -> ParsecT s u m a

Therefore, you already need a generic token parser, which you can create with makeTokenParser. The latter has the type:

makeTokenParser
  :: Stream s m Char =>
     Text.Parsec.Token.GenLanguageDef s u m
     -> Text.Parsec.Token.GenTokenParser s u m

It takes a language definition and returns a token parser. Since you don't have any specific language in mind, you can use emptyDef from Text.Parsec.Language. Note that whiteSpace also takes a GenTokenParser. And last, in this setup you will end up with ds :: [Char], therefore you need to use digitToInt from Data.Char before you can actually sum your digits:

import Text.Parsec
import Text.Parsec.Token (lexeme, makeTokenParser, whiteSpace)
import Text.Parsec.Language (emptyDef)
import Data.Char (digitToInt)

lexer = makeTokenParser emptyDef

mainParser  = do{ whiteSpace lexer
     ; ds <- many (lexeme lexer digit)
     ; eof
     ; return (sum . map digitToInt $ ds)
     }

main = do
  putStrLn "Please give some digits (whitespaces are ignored)"
  line <- getLine
  case parse mainParser "" line of
    Right n -> putStrLn $ "Sum of digits is " ++ show n
    Left  _ -> putStrLn $ "Couldn't parse your line"

Example output:

*Main> :main
Please give some digits
7 8 91 72 3945 01 92
Sum of digits is 67

*Main> :main
Please give some digits
abc 1
Couldn't parse your line