Dataweave - Globally Unique Array Elements

Question

Hi I have a dataweave question, I have the sample input and I want to remove the 2nd object in the 2nd array. In short, records should be unique globally. I tried distinct and filter, however filter remove both, distinct doesn't change anything.

Input

[
  {
    "id": 1,
    "books": [
      {
        "bookid": "1234",
        "title": "C#"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  },
  {
    "id": 2,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  }
]

Output

[
  {
    "id": 1,
    "books": [
      {
        "bookid": "1234",
        "title": "C#"
      },
      {
        "bookid": "5678",
        "title": "Java"
      }
    ]
  },
  {
    "id": 2,
    "books": [
      {
        "bookid": "4321",
        "title": "VB"
      }
    ]
  }
]

Answer 1

I made an assumption that we would be picking the first unique title and discard the remaining. Below I broke down your problem into 4 steps. First, I am adding id with the books, then remove duplicates using distinctBy, then group filtered list by id and finally, write your desired output. I am curious to see other solutions :)

%dw 2.0
output application/json
var globalList =  payload flatMap ((item) -> (
    item.books map (b) -> {id: item.id,(b)}))
    distinctBy ((item) -> item.bookid)
    groupBy ((item) -> item.id) 
---
  keysOf(globalList) map ((item) ->
    {
        id: item,
        books: globalList[item] map ((book) -> book - 'id')
    
} )