Search code examples
file-uploadfilesystemsbackupchecksumrsync

Looking for incremental backup algorithm

I am looking for a solution for files and directories synchronization over WAN.

It must:

  1. support Windows and Linux based OS
  2. support both textual and binary files
  3. be incremental
  4. resume broken transfers
  5. nice-to-have: support cascaded or distributed backups in order to prevent single point of failure problem.

I have read about rsync and its implementations. However, the rsync could be relatively slow due to its file list parsing algorithm and also in case of binary files the "incremental" backup is a full backup in most cases which can cause heavy traffic. (I am aware of the fact that in the worst case there is no way to sync binary files incrementally but only by full backup like in case of backing up files that are compilation products). I have also checked several related questions here (the most useful was this answer)

So, my question is: are there any (new?) useful algorithms/techniques/heuristics that can be applied in order to solve the task described above?

Any tips would be appreciated.

Solution

  • I recommend you RMAN backup algorithm. It has the following advantages over the traditional incremental backups:

    1. Ability to Recover one block of datafile
    2. Ability to automatically backup CONTROLFILE and SPFILE
    3. Ability to delete the older ARCHIVE REDOLOG files, with the new one's automatically.
    4. Ability to perform backup and restore with parallelism.
    5. Ability to report the files needed for the backup.
    6. Ability to resume the failed backup (without starting from beginning).
    7. Much faster when compared to other TRADITIONAL backup strategies.