Search code examples
jsonapache-nifijolt

NiFi | How to replace text based on length of field in JSON records


I have data coming into NiFi in JSON format. Below is the sample

{
  "company_name": "mycompany",
  "companytyp": "producer",
  "Welcomemes": "welcome_message1",
  "logo": "//somepath/to/logo URL "
}

I have been trying to apply a JOLT transformation for the following condition:

  • If the length of logo field is more than 10 characters then replace it with blank value
  • If the length of logo field is less than or equal to 10 characters then keep the provided value

I have not been able to achieve this. Would really appreciate some help & guidance. I am open to using any other processor too but the preference will be JOLT transformation

Input 1:

{
  "company_name": "mycompany",
  "companytyp": "producer",
  "Welcomemes": "welcome_message1",
  "logo": "This logo value is more than 10 characters "
}

Expected Output 1

{
  "company_name": "mycompany",
  "companytyp": "producer",
  "Welcomemes": "welcome_message1",
  "logo": ""
}

Input 2:

  "company_name": "mycompany",
  "companytyp": "producer",
  "Welcomemes": "welcome_message1",
  "logo": "short logo"
}

Expected Output 2

{
  "company_name": "mycompany",
  "companytyp": "producer",
  "Welcomemes": "welcome_message1",
  "logo": "short logo"
}

Solution

  • I couldn't figure out how to put in an empty string but this spec will replace those logo fields with a single space:

    [
      {
        "operation": "modify-overwrite-beta",
        "spec": {
          "logo": "=substring(@(1,logo),0,11)",
          "logo_size": "=size(@(1,logo))"
        }
      },
      {
        "operation": "shift",
        "spec": {
          "logo_size": {
            "11": {
              "# ": "logo"
            },
            "*": {
              "@(2,logo)": "logo"
            }
          },
          "logo": null,
          "*": "&"
        }
      }
    ]