I have CSV files in a container in Data Lake and I am trying to transform the attribute column(which is in json) in the files into multiple columnns. When I try using Data flows to do that, It recognizes the attribute column as string and I am unable to change it to array so that I can either flatten or create derived columns out of it. Am i doing something wrong here CSV File Format:
ID | Name | Attribute |
---|---|---|
123 | Test | {"Referrer URL":null,"Query Parameters":"topics"} |
456 | Test2 | {"Referrer URL":"www.google.com","Query Parameters":"WebTopics"} |
After following guidance of Mark and Joseph: Issues faced while using parsing function and removing spaces from keys in json objects in those columns
Update:
Hi@Hammad Hassan Khan, due to your Attribute
column contains "," character, so I edit it as follows. And use Pipe (|) as Column delimiter in the source:
Attached to the answer given by @Mark Kromer MSFT. Yes we can use Parse transformation in mapping data flow to achieve that. But the Parse
activity does not support JSON objects whose keys contain space characters. Therefore, we need to replace space characters.
I created a simple test for this. And the result is as follows:
In DerivedColumn1
activity. Select Attribute
column, enter expression replace(Attribute,' ','' )
to replace spaces in keys. Because in the next Parse1
activity, it does not support keys contain space characters.
The data preview is as follows:
In Parse1
activity, select Attribute
column, enter Attribute
as Expression, enter (ReferrerURL as string,QueryParameters as string)
as Output column type.
The data preview is as follows: