Search code examples
httprsyncprocessing-efficiency

synchronization over http: rsync versus normal upload

I'm running file synchronization over HTTP. Both sides implement rsync. When synchronizing, for uploading I have two choices:

  1. use a simple post request if:
    • the file to be uploaded does'nt exists on the remote side.
    • the file exists and is bigger than a certain value M.
  2. else : perform rsync over get requests.

My question is: How can I determine the perfect value of M.

I'm certain that for a certain file size, performing simple upload is faster than performing rsync steps . Especially for multiple files.

Thanks

Solution

  • If you're using rsync correctly, I'd bet that it's always faster, especially with multiple files.

    Rsync is specially built to check differences between directory trees and update the target directory incrementatlly.

    The following is a one-liner to keep in mind whenever you need to sync two directory trees.

    rsync -av --delete /path/to/src /path/to/target

    (also works over SSH, if necessary.)

    Only keep in mind that rsync is picky about trailing slashes on directory paths.