DelimitedText (csv) dataset encodes filepaths to urlencoding format automatically and successive copy activity cannot find blobs

Question

I have a data factory which is having a dataflow activity and successive copy activity inside a foreach loop. The dataflow pulls folder paths and filenames together from source container in bob storage and while passing to foreach loop it creates urlencoding of the folderpaths and filenames. For example x/y/z/sample doc.pdf becomes x/y/z/sample%20doc.pdf m/n/p/sample[1].pdf becomes m/n/p/sample%5B1%5D.pdf

Because of this copy activity throws exception The required blob is missing. If I manually remove the encoding it can find blobs. please suggest what is the solution for this issue ?

I have tried changing datasets encoding from Default UTF 8 to other encodings but it did not help.

Answer 1

You can use decodeUriComponent(<input_string>) for getting back the original file path.

Create dataset parameters for the file names of the source and sink datasets and give the below expressions for those in the copy activity inside For-Each.

Source filepath - @decodeUriComponent(item().filepath)
Sink file name - @decodeUriComponent(item().targetpath)

For sample, I am showing these conversions using set variable activities.

Results:

Source file path:

Target file name: