Search code examples
haskellhaskell-pipes

Join two consumers into a single consumer that returns multiple values?


I have been experimenting with the new pipes-http package and I had a thought. I have two parsers for a web page, one that returns line items and another a number from elsewhere in the page. When I grab the page, it'd be nice to string these parsers together and get their results at the same time from the same bytestring producer, rather than fetching the page twice or fetching all the html into memory and parsing it twice.

In other words, say you have two Consumers:

c1 :: Consumer a m r1
c2 :: Consumer a m r2

Is it possible to make a function like this:

combineConsumers :: Consumer a m r1 -> Consumer a m r2 -> Consumer a m (r1, r2)
combineConsumers = undefined

I have tried a few things, but I can't figure it out. I understand if it isn't possible, but it would be convenient.

Edit:

I'm sorry it turns out I was making an assumption about pipes-attoparsec, due to my experience with conduit-attoparsec that caused me to ask the wrong question. Pipes-attoparsec turns an attoparsec into a pipes Parser when I just assumed that it would return a pipes Consumer. That means that I can't actually turn two attoparsec parsers into consumers that take text and return a result, then use them with the plain old pipes ecosystem. I'm sorry but I just don't understand pipes-parse.

Even though it doesn't help me, Arthur's answer is pretty much what I envisioned when I asked the question, and I'll probably end up using his solution in the future. In the meantime I'm just going to use conduit.


Solution

  • I think something is wrong with the way you are going about this, for the reasons Davorak mentions in his remark. But if you really need such a function, you can define it.

    import Pipes.Internal
    import Pipes.Core
    
    zipConsumers :: Monad m => Consumer a m r -> Consumer a m s -> Consumer a m (r,s)
    zipConsumers p q = go (p,q) where
      go (p,q) = case (p,q) of 
         (Pure r     , Pure s)      -> Pure (r,s)
         (M mpr      , ps)          -> M (do pr <- mpr
                                             return (go (pr, ps)))
         (pr         , M mps)       -> M (do ps <- mps
                                             return (go (pr, ps)))
         (Request _ f, Request _ g) -> Request () (\a -> go (f a, g a))
         (Request _ f, Pure s)      -> Request () (\a -> do r <- f a
                                                            return (r, s))
         (Pure r     , Request _ g) -> Request () (\a -> do s <- g a
                                                            return (r,s))
         (Respond x _, _          ) -> closed x
         (_          , Respond y _) -> closed y
    

    If you are 'zipping' consumers without using their return value, only their 'effects' you can just use tee consumer1 >-> consumer2