Search code examples
shellunixcattail

tail multiple CSV files to new file?


i have 2 files: testfile1.csv and testfile2.csv.

testfile2.csv:

time,topic3,topic4
2015-10-01,40,50
2015-10-02,45,55
country,uk,uk

testfile1.csv:

time,topic1,topic2
2015-10-01,20,30
2015-10-02,25,35
country,usa,usa

i want to combine them into 1 big file, with test file1 being duplicated and having only 1 header so i did this:

cat test1/testfile1.csv > testfile3.csv
tail -n +2 test1/testfile1.csv  test2/testfile2.csv >> testfile3.csv 

The output looks right except for indicators to show which set of data came from which file:

time,topic1,topic2
2015-10-01,20,30
2015-10-02,25,35
country,usa,usa
==> test1/testfile1.csv <==
2015-10-01,20,30
2015-10-02,25,35
country,usa,usa

==> test2/testfile2.csv <==
2015-10-01,40,50
2015-10-02,45,55
country,uk,uk

How do i remove the indicators? am i writing the tail -n + 2 part wrong? When i use tail -n +2 test2/testfile2.csv >> testfile2.csv the output looks good but i dont want to do it manually file by file.

Expected output:

time,topic1,topic2
2015-10-01,20,30
2015-10-02,25,35
country,usa,usa
2015-10-01,20,30
2015-10-02,25,35
country,usa,usa
2015-10-01,40,50
2015-10-02,45,55
country,uk,uk

Solution

  • From the manual (man tail):

         -q      Suppresses printing of headers when multiple files are being examined.
    

    Therefore, the command would be:

    tail -qn +2 test1/testfile1.csv  test2/testfile2.csv >> testfile3.csv