Search code examples
azure-data-factory

How to change the column data types of a data flow in Azure Data Factory

I'm running a dataflow activity in ADF which uses a REST API response as its source. The response has integer values for some key values like the following example.

"ValuatedBy": 0,

These values are interpreted as Booleans by ADF and displays as follows in the inspect tab for the source.

  1. How to change these values to have the correct data type?
  2. Why does this happen?

Wrong interpretation of the data types

I tried Casting but casting activity won't help as Boolean casted to String displayed as "true" or "false".

Solution
    • You can change the type of the column manually in source transformation.
    • Click the Projection tab in the source transformation of data flow.
    • In the column name which contains ValuatedBy field, select Define Complex Type.

    img1

    • In dataflow expression builder, change the type of ValuatedBy field from boolean to Integer or any other required type. Then click Save and Finish.

    gif1

    • Once changed, ValuatedBy is reflected as Integer type in Inspect tab. img2
    1. Why does this happen?

    This happens because ADF automatically infers the data types of the columns in the source based on the first few rows of data. If the first few rows of data contain only 0s and 1s, ADF may infer that the column as a Boolean column.