Search code examples
haskellstatemonadsmonad-transformersmegaparsec

State with Megaparsec ParsecT is not backtracking


I have a parser defined as a slightly more complicated version of the following:

data X = X { getX :: State ([Int], [X]) Bool }
type Parser = ParsecT Void String (State ([Int], [X]))

The idea is that I can build up a stack of actions I want to do to my state (the [Int]) and then execute them in any order or whenever I want, depending on the circumstances:

-- Run the first state in the list.
executeOne :: Parser Bool
executeOne = do
  s@(_, fs) <- get
  let (r, s') = (flip runState s) . getX . head $ fs
  put s'
  return r

For example, the action executed may reorder the stack of actions or modify the [Int].

Design decisions aside (I'm sure there are better ways to do this), it seems that backtracking with try is not working with the state. Specifically, the state of the ParsecT will backtrack, but the inner state (the [Int] and [X]) won't. Why is this? Am I misusing ParsecT or is the strange recursive X business screwing everything over? Do I need to use Control.Monad.State.Strict instead?

Edit: To answer a commenter's question about an example X, here's one:

useless :: X
useless = X $ do
  (vs, xs) <- get
  if length vs >= 10
  then do { put (vs, tail xs) ; return True }
  else do { put (vs ++ vs, xs) ; return False }

useless doubles our [Int] if it has less than ten elements, and returns False. If it does have ten or more elements, it removes itself and returns True. The power in having X be recursive is that it can choose whether or not to remove itself after it's done.


Solution

  • The problem was the order of the transformers in the monad stack.

    Informally speaking, a transformer can't "cancel" or "nullify" the effects of the base monad it transforms. For example, StateT over ExceptT loses its state on failure, while ExceptT over StateT doesn't. (This is also the reason why there can't be an IOT transformer: how to nullify a effect that has already escaped into the world?)

    Here, it means that the inner State will survive any backtracking of the parser. The solution is to put StateT above the parser monad, not below.

    This would seem to require lift calls for all the parser functions, because the parser is not the outermost monad now. Fortunately, the lifts aren't needed because StateT s Parser is an instance of MonadParsec, which auto-lifts all parser operations.