Search code examples
haskellfoldhaskell-lensaesonlenses

How to do mapMaybe using lenses

I am using wreq on the github api to get a list of files in a repository. I include this for completeness sake. This isn't about doing the web request:

    let
        myOpts = defaults
          &  header "Accept" .~ ["application/vnd.github.raw"]
          &  header "X-GitHub-Api-Version" .~ ["2022-11-28"]

        url = "https://api.github.com/repos/rubenmoor/learn-palantype/git/trees/main?recursive=1"

    liftIO (try $ getWith (myOpts & auth .~ mAuth) $ Text.unpack url) <&> \case
      Left (HttpExceptionRequest _ content) -> Error 500 $ Text.pack $ show content
      Left (InvalidUrlException u msg) -> Error 500 $ "Url " <> Text.pack u <> " invalid: " <> Text.pack msg
      Right resp -> -- ...

The resp is a JSON-encoded and looks something like this (only in reality a lot more files):

{
  "sha": "7fd9d59c9b101261ca500827eb9d6b4c4421431b",
  "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/trees/7fd9d59c9b101261ca500827eb9d6b4c4421431b",
  "tree": [
    {
      "path": ".github",
      "mode": "040000",
      "type": "tree",
      "sha": "eb21b416a406ebae963116911afd3cd0994132ce",
      "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/trees/eb21b416a406ebae963116911afd3cd0994132ce"
    },
    {
      "path": ".gitignore",
      "mode": "100644",
      "type": "blob",
      "sha": "a47bd530c4b8677af24b291b7c401202ca1170d4",
      "size": 186,
      "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/blobs/a47bd530c4b8677af24b291b7c401202ca1170d4"
    },
    {
      "path": "static.nix",
      "mode": "100644",
      "type": "blob",
      "sha": "fcac7837dc13cce9368517ba8ba49a00d5b76734",
      "size": 353,
      "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/blobs/fcac7837dc13cce9368517ba8ba49a00d5b76734"
    },
    {
      "path": "cms-content/SystemDE/EN/Introduction.md",
      "mode": "100644",
      "type": "blob",
      "sha": "25b2be5dd3fd3d2a7a1c8fc95ed7e9623e7bd5c6",
      "size": 2670,
      "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/blobs/25b2be5dd3fd3d2a7a1c8fc95ed7e9623e7bd5c6"
    },
    {
      "path": "cms-content/SystemDE/EN/Pattern Overview.md",
      "mode": "100644",
      "type": "blob",
      "sha": "c34f97e9666e56ec12e554afc7f684e9666b74fd",
      "size": 18,
      "url": "https://api.github.com/repos/rubenmoor/learn-palantype/git/blobs/c34f97e9666e56ec12e554afc7f684e9666b74fd"
    }
  ],
  "truncated": false
}

Now I can use Data.Aeson.Lens to go into the json structure like this:

resp ^. responseBody . key "tree" ^.. -- ???

Now comes the tricky part. I am only interested in markdown files inside a directory called "cms-content", or subdirectories thereof. Files have the value "blob" at the key "type". And for those files, I want their full path w/o the filetype extension. So, given the example JSON, I am looking for this result

["SystemDE/EN/Introduction", "SystemDE/EN/Pattern Overview"] :: [Text]

I think of mapMaybe and can define a suitable function like that:

maybeCMSFile :: Text -> Text -> Maybe Text
maybeCMSFile strType strPath | strType == "blob" =
    case Text.stripPrefix "cms-content/" strPath of
        Nothing  -> Nothing
        Just suf -> Text.stripSuffix ".md" strPath
maybeCMSFile _ _ = Nothing

The arguments for maybeCMSFile are values for specific keys of the objects in the JSON array:

\o -> maybeCMSFile (o ^. key "type" . _String) (o ^. key "path" . _String)

But instead of converting the JSON array into a list (_Array from Data.Aeson.Lens gets me there) and running mapMaybe maybeCMSFile . Vector.toList, I am looking for a way to use lenses to the same end. I can simplify the problem for myself quite a bit by breaking things down in simpler steps:

  1. filter for the key/value "type": "blob"
  2. filter for the suffix ".md" in the value at key "path"
  3. extract the filepath without the suffix "cms-content" and without the prefix ".md"

But of course I am wondering, if this can all be done just combining the right lenses.

Let me add that I am well aware that this question is awfully specific. Personally, I learned my way around lenses by these kind of examples. I still have troubles reading the type signatures and making sense of the lenses (and prisms) with the help of the documentation on hackage alone.

Solution

  • A minor variation on your not-compiling code:

    resp ^.. responseBody . key "tree" . _Array . each
  . filteredBy (key "type" . _String . only "blob")
  . key "path" . _String
  . filtered (\str -> "cms-content/" `Text.isPrefixOf` str && ext `Text.isSuffixOf` str)
  . folding (Text.stripPrefix "cms-content/" >=> Text.stripSuffix ext)