Search code examples
dynamicfixed-widthderived-columnazure-data-factory

ADF DataFlow Activity how to create dynamic derived column


I have input fixed width txt file as source.

test file sample below

column_1
12ABC3455
13XYZ5678

How to build dynamic column pattern to produce derived columns.

column Name : empId -> substring(column_1,1,2)

derive Column setting

I can hardcode the empid in & substring(column_1,1,2) in expression.

but i need to make it dynamic with the JSON input to derive dynamic derived columns with column pattern.

Below sample JSON input parameter.

My input JSON formatted parameter

[
   {
      "colname": "empid",
      "startpos": 1,
      "length": 2
   },
   {
      "colname": "empname",
      "startpos": 3,
      "length": 3
   },
   {
      "colname": "empSal",
      "startpos": 6,
      "length": 4
   }
]

help me to build the column pattern with the json input


Solution

  • I tested many times and can't achieve that.

    Just per my experience, I'm afraid to tell you that it's impossible to to that in Data Factory actives or Data Flow with json parameter.