Duplicate ids are coming into result with some null values

Question

I have the below JSON where I want to filter out unique product.id into the array variable.

I am using the below query but the result is not coming in the proper way.

productIds=($(jq -r .items[].product.id $Outputfile))

Actual output: 123 123 123 456 456 null null

Expected output: 123 456

{
  "documentType": "product",
  "items": [
    {
      "ResourceId": null,
      "product": {
        "href": null,
        "id": "123",
        "mainId": "qwe345",
        "primaryId": "5298"
      },
      "lastModifiedBy": "test",
      "quantity": null,
      "effectiveDate": null,
      "extensions": null
    },
    {
      "product": {
        "href": null,
        "id": "123",
        "mainId": "qwe678",
        "primaryId": "5643"
      },
      "lastModifiedBy": "test",
      "quantity": null,
      "effectiveDate": null,
      "extensions": null
    }
  ],
  "createdBy": "test",
  "createdOn": "2021-10-05",
  "currentSeqNum": 2
}

Then I want to encode those value into base64 format and then concatenate with some constant value. I am not able to concatenate with constant as well and store in same variable inside loop.

Can you please help me with above two issues.

Answer 1

You can use unique after applying array conversion and then join such as

productIds=$(jq -r '[.items[].product.id] | unique | join(" ")' $Outputfile)

Demo