Search code examples
bashsedgnu-parallel

sed adds string at the end of each new line


I am currently editing multiple CSVs to add 2 columns ORIGIN_FILE and WRITE_DATE. The problem is that the string is always added after a new line for some reason.

Here is an example of my input csv:

PRODUCT, QUANTITY, PRICE
ITEM1, 4.8, 380.33
ITEM2, 2.3, 120.50
ITEM3, 1.5, 210.11

But everytime I run my bash command below:

find $target_dir -type f -name "*.csv" | parallel sed -i '\'1s/$/,ORIGIN_FILE,WRITE_DATE/; 2,$s/$/,myfilename.csv,2022-11-24/\'' {}'

This is the output I get:

PRODUCT, QUANTITY, PRICE, ORIGIN_FILE, WRITE_DATE
ITEM1, 4.8, 380.33
myfilename.csv, 2022-11-24
ITEM2, 2.3, 120.50
myfilename.csv, 2022-11-24
ITEM3, 1.5, 210.11
myfilename.csv, 2022-11-24

It always creates a new line after the second row and add my string there.

I'm hoping to get this output:

PRODUCT, QUANTITY, PRICE, ORIGIN_FILE, WRITE_DATE
ITEM1, 4.8, 380.33, myfilename.csv, 2022-11-24
ITEM2, 2.3, 120.50, myfilename.csv, 2022-11-24
ITEM3, 1.5, 210.11, myfilename.csv, 2022-11-24

Hoping someone can help me spot what I'm doing wrong.

I'll appreciate if someone can just help me with the sed command.


Solution

  • I was able to solve the problem using the command below:

    sed  '1s/$/,ORIGIN_FILE,WRITE_DATE/; 2,$s/\(\s\)\?$/,myfile.csv,2022-11-24/gmi' myfile.csv
    

    I had to add \(\s\)\?$ in the matching regex because it looks like my input file is having an invisible whitespace which is not a \n character but it pushes my data down to the next line. The reason I know that it is not \n because I cannot match it in the regex using \n$.

    I'm still puzzled on what's happening here. But I'm really glad I was able to solve it with all your help and suggestions.

    Thanks a lot!