Search code examples
excelcsvheadwc

how to make csv file split the lines appropriately?

I have an enormous csv file, about 100 M. I try to do a head on it, and it spits out the entire csv, regardless of the number of lines I ask for.

head data.csv AND head -1 data.csv give the same result, i.e. the entire file.

I notice later that there are no lines in the file at all:

wc -l data.csv
0 data.csv

I open the csv in excel, and it looks as it should. And, I save it again as a csv.

How do I get the lines back in my csv file?

Solution

  • This shows the invisible character ^M: 

    cat -vets data.csv >> new_data.csv

    Then, this replaces the invisible character, ^M with the newline character $\n:

    sed -i .copy 's/\^M/\'$'\n/g' new_data.csv

    then when you head new_data.csv, you see that the lines are separated appropriately.