So, I have multiple files of the form
filename_date.csv
in a folder, for example something like
filename_2010_01_01.csv
that files contains, say... 1000 rows with a format like this
37845287|2364532467|237849853
04568750|89345735385|94573857
But I have 365 files like this for the whole year, one for each day. What I'd like to do is to make some kind of bash? script maybe that takes the date from the filename and appends it to the end of each of the x lines in each file so the result would be something like
filename_2010_01_01.csv
37845287|2364532467|237849853|2010_01_01
04568750|89345735385|94573857|2010_01_01
filename_2010_01_02.csv
438574|563546567|2342523525|2010_01_02
326345452|456454763|7856763|2010_01_02
So far what I've tried:
I tried with this
sed 's/$/|2010-01-01/' filename_2010-01-01.csv > filename_2010-01-01.csv
that works for a single file.
Now I'm trying the following loop
FILES=~/Desktop/2010/byday/*
for f in $FILES
do
filename=$f | cut -d'/' -f7-
done
Once I have the name of the file there, I can just add the sed command within the loop but I'm having trouble on making the filename
var to take the value of the command
If I just echo the command within the loop it does get me the filename, that command is only to strip the path from the filename.
UPDATE:
Inside the loop
filename=`basename $f`
that shorter, cleaner and the variable actually gets the value
UPDATE 2:
This loop
FILES=~/Desktop/2010/byday/*
for f in $FILES
do
fname=`basename $f`
fname=${fname%.csv}
fname=${fname##filename_}
echo $fname
sed 's/$/|$fname/' filename_$fname.csv > filename__$fname.csv
done
Actually outputs correctly the date BUT there's something in the sed command that deletes the content of the file
UPDATE 3:
With this Im almost there
FILES=~/Desktop/2010/byday/*
for f in $FILES
do
fname=`basename $f`
fname=${fname%.csv}
fname=${fname##partitioned.csvmerged_}
sed 's/$/|$fname/' partitioned.csvmerged_${fname}.csv > $fname.csv
done
It writes a new file with just the date as filename (which is good) but Im not sure how should I interpolate the var there since with this the output inside the file is
438574|563546567|2342523525|$fname
326345452|456454763|7856763|$fname
You are overwriting the same file that you are reading. As soon as the shell parses the command, it starts truncating the file and then, with the output file prepared, it starts the sed
command, that finds an empty file. Your problem will be solved by avoiding the redirection and using the -i
parameter in sed
:
FILES=~/Desktop/2010/byday/*
for f in $FILES
do
fname=$(basename "$f" .csv)
fname=${fname##filename_}
echo "$fname"
sed -i "s/\$/|$fname/" "filename_$fname.csv"
done
I also added a few minor changes, like using the already present basename
to get rid of the extension, quoting variables and solving your next problem: not having $fname
expanded due to the use of single quotes.