Search code examples
azure-data-factorygoogle-cloud-dataflow

ADF Mapping Dataflow passing parameter values

I am passing in a parameter of type string into an ADF Data flow. I have a derivied column and I am trying to set that column value to the parameter value. However, I keep getting an error Column 'ClientA' used in expression is unavailable or invalid. For column it is showing the value of the parameter. In debug, I have to wrap the parameter value in single or double quotes for it to work. I do not know how to make the dataflow derived column value interpret the parameter as a string inside the expression. I've tried concatenating the parameter with stings or double quotes but it still doesn't work. ParameterName: ClientName ParameterValue: ClientA ParameterType: String

I also have another derived column of type int, but that one is mapping without issue.

enter image description here

Solution

  • You need to use the dataflow parameters in the derived column as show below.

    First create the required parameter in the dataflow with required type.

    enter image description here

    Give the parameter to the columns in the derived column transformation.

    enter image description here

    If you want to test the data preview inside the dataflow, go to the debug settings and provide the value for the parameter.

    enter image description here

    If you want to pass the parameter value from the pipeline, you need select pipeline expression here and give your string as shown below. You can update the value like concatenation with any other strings here using pipeline expression.

    enter image description here

    Now, run the dataflow from the pipeline and it will show the parameter value in the dataflow activity input.

    enter image description here

    I have opted for the sink cache and you can see the sink results in the dataflow activity output.

    enter image description here