Search code examples
linuxbashbash4

Merging directories and joining inner files with bash


I have bunch of directories following this structure:

\directories\
 |---\directory1\
 |      |---file1.txt
 |      |---file2.txt
 |
 |---\directory2\
 |      |---file1.txt
 |      |---file3.txt
 |
 |---\directory3\
 |      |---file4.txt
 |      |---file2.txt

I want to merge the directories and files so it ends up like this:

\directories\
 |---\directory1\
 |      |---file1.txt
 |      |---file2.txt
 |
 |---\directory2\
 |      |---file1.txt
 |      |---file3.txt
 |
 |---\directory3\
 |      |---file4.txt
 |      |---file2.txt
 |
 |---\mergeddata\
 |      |---file1.txt (from dir1 and dir2)
 |      |---file2.txt (from dir1 and dir4)
 |      |---file3.txt (from dir2)
 |      |---file4.txt (from dir3)

I am awful with bash and been trying quite a few things but... not getting any good results.

Looking forward to some help!


Solution

  • You didn't say how you want the files merged, or in what order. I will guess "concatenated, with directory1 appearing before diretory2, and directory2 before directory3"?

    The following script shows a straightforward way to do this, without relying on fancy substitutions:

    cd directories
    mkdir mergeddata
    for I in directory1 directory2 directory3 ; do   # replace with your actual directory list
        for F in "$I"/* ; do
            B=$(basename "$F")
            cat "$F" >> "mergeddata/$B"
        done
    done
    

    Edit: I added some quotes, in case any of your filenames end up with space characters or other inconvenient white space.