Search code examples
parametersazure-data-factory

How to set default parameter with data type object in ADF

I have a generic pipeline where I want the cluster sizes to be configurable. The idea is to use an optional parameter where I can set a default pipeline in the generic pipeline, so I don't have to change many other pipelines that reference this generic pipeline. The parameter definition is shown here below. Parameter overview of generic pipeline

We now want to use this Object parameter in order to select the cluster size corresponding to our current environment. This is done as follows:

Use parameter in generic pipeline

This goes well if I pass an explicit value for the Parameter from the outer (calling) pipeline, like so: Call generic pipeline with explicit value configured.

I also want other (unchanged) pipelines that call the generic pipeline to continue running with a default value. Those are called as follows:

Call generic pipeline without explicit value configured

but that results in the following error:

enter image description here

Please note: it is not possible to use Dynamic Content to set the default value for a Pipeline Parameter of type Object. Which adaptations should I make in order to correctly set the default value for clusterSizes as a json object?

Solution

  • I tried the above scenario with the below JSON as object type parameter in a child pipeline.

    {"name":["Rakesh"],"Age":[22],"nickname":["Laddu"],"luck":[]}

    I have used @pipeline().parameters.param1[variables('one')] expression in the child pipeline and called it from Parent pipeline.

    enter image description here

    And I got the same error.

    enter image description here

    This is because, when we are not passing any values for the child pipeline parameters from parent pipelines and expecting to take default parameter values for it, it is taking the type of that value as a String.

    Input for the Execute pipeline activity.

    enter image description here

    You can see it took the value as string even though we defined it as an object type.

    When I tried with other data types, it took correct types for int and bool types.

    enter image description here

    enter image description here

    But, for the Object, array and float types it took type as string only. This might be a bug in ADF.

    As you want the other pipelines to continue with default values, changing the generic pipeline might be the workaround in this case.

    Which adaptations should I make in order to correctly set the default value for clusterSizes as a json object?

    In your generic pipeline, when using that parameter convert that to object with the below expression.

    @json(string(pipeline().parameters.param1))[variables('one')]

    With this expression, the pipelines which requires default values will be converted to JSON object and the pipelines which uses the dynamic content parameter values will be converted to string and again converted to JSON object.