Search code examples
azure-data-factory

Custom Row Delimiter in Azure Data Factory (ADF)


I have a CSV file that terminates a row with Comma and CRLF. I set my dataset to ",\r\n" but when I ran the pipeline, it won't accept this, thinking it's multiple values in the delimiter... If I don't put the comma in the dataset row delimiter, when pipeline runs, it thinks that there's an unnamed header. Is it possible in ADF to have this combination as a delimeter (comma + crlf) - ",\r\n"?

FirstName,LastName,Occupation,<CRLF Char>

Michael,Jordan,Doctor,<CRLF Char>


Solution

  • Update:
    When running the copy activity, I encountered the same problem as you.

    1. Then I selcet Line feed(\n)as Row delimiter at Source. enter image description here

    2. Add Column mapping as follows:
      enter image description here

    3. When I run debug, the csv file was successfully copied into Azure SQL table. enter image description here enter image description here


    I created a simple test. Do you just want ADF to read 3 columns?

    1. This is the origin csv file.

    kayeesp

    1. In ADF, If we use default Row delimiter and Column delimiter settings, select First row as header.
      enter image description here

    2. We also can select Edit and enter \r\n at Row delimiter field.
      enter image description here

    3. You can import schema here. enter image description here