Search code examples
unixoptimizationcp

Fastest way to copy a large folder in unix


I'm trying to copy a large folder around 1-5gb which contains sub directories with files that range from 1kb-35mbs. I tried using 'cp' but it takes over 10 minutes. Copying and pasting in windows seems to do the job faster.

I'm new to unix and I was wondering if there was a faster alternative or cp, or a way to optimize the way it works. I've read up on buffers but I am so confused as to how they work (so I decided not to use it).

What I am trying to do: I need to transfer files from my H drive to a network drive elsewhere (also delete the files from the network drive so that the new files can be copied to it). It's a really straight forward task which I have working. Just takes about two decades to finish.

P.S. Not sure if I have been told wrong, but working in unix is generally faster than windows right?


Solution

  • You could try using tar or a compressor like zip or gzip to put the whole directory structure into a single archive file, copy the archive file, then unarchive it at the destination. I doubt that would save you time overall, but at least the copying step would be faster.

    tar -cvf folder.tar /path/to/original/folder
    cp folder.tar /path/to/destination
    cd /path/to/destination
    tar -xvf folder.tar
    

    -c create
    -v verbose
    -f file name is... (user put it following)