Search code examples
azureazure-data-factorygoogle-cloud-dataflow

ADF Data flow - data map or translate


Using a data factory, I have a data flow that reads the data from the Rest endpoint A field comes int he form

{
  "Field1":2007,
   "Field2":2005
}

I need to convert the values to a word according to specific definition:

2005 = "AAA"
2007 = "BBB"

So that in the sink it would produce the result:

{
  "Field1":"BBB",
  "Field2":"AAA"
}

These fields are NOT required, so they may or may not apear in the payload returned from the Rest service.


What transformation should I use ?
The dictionary is small - 2-3 values, so anything manual is acceptable.


Solution

    • Add a Derived Column transformation to your data flow. In the Derived Column transformation, add the column for each field that you want to convert.

      For example, if you want to convert "Field1" and "Field2", you would add these two columns.

    • For each column, use the following expression to convert the numeric value to its corresponding word value:

      iif(Field1 == 2005, 'AAA', iif(Field1 == 2007, 'BBB'))

    • Replace "2005" and "2007" with the numeric values that you want to convert to their corresponding word values.

    • Repeat this process for each field that you want to convert.

    • Finally, use a Select transformation to select only the columns that you want to include in the output. This will remove any columns that were not converted.