I have a file with 4 million of lines, every line ends with the char $, but I mistakenly add a new line after the the line delimiter while scraping a website, so right now it is looking like this:
fist name, last name, phone, address, postal code, city, region,$
$
fist name, last name, phone, address, postal code, city, region,$
$
the new line '$' only shows up of course if I use :set list, but I'm trying to use this file for a bulk insert in mysql and I'm having problems with it now.
I would like to change the file to:
fist name, last name, phone, address, postal code, city, region,$
fist name, last name, phone, address, postal code, city, region,$
How can I do this? with sed or awk or even vi ? looked up around and what I found is not really applying to this case.
please don't take in consideration the extra empty line shown above.
Thanks in advance
To remove blank lines with sed:
sed -i '/^$/d' yourfile.csv
To remove lines consisting of a single $:
sed -i '/^$$/d' yourfile.csv
Most versions of sed support the -i switch; if yours does not you will need e.g. sed '/^$$/d' yourfile.csv > newfile.csv.
Removing blank lines with white space is more complicated. This usually works:
sed '/^ *$/d' yourfile.csv
If this is not sufficient, try checking also for tabs. For older sed's, this will work:
sed '/^[ X]*$/d' yourfile.csv
where X here a tab, entered via Control-V Tab.
Newer sed's will take a [ \t\r]* or \s* or [[:space:]]*, sometimes requiring a -E switch.