Search code examples
duplicityduplicity-backup

Does duply back up the state of a file when it is read or when duply command is started?

I want to back up 100 TBs of data. The backup task should take 30 days given my computation power and bandwidth. But the data are not stagnant. During the progress of back up, data will be modified/created/deleted within the source directory.

Question is will duply/duplicity back up the state of files when a file is read or when backup is initiated?

Solution

  • duplicity will use the file state at the point in time when the file is processed during the backup.

    Note: as a user application duplicity is not capable to enforce file system consistency, meaning if the file is readable, but currently open in another application and written only partially, this inconsistent state will be backed up.

    Suggestions

    1. use a files system that is snapshot capable and backup those
    2. stop services/software that might write data to be backed up, to retrieve a consistent state beforehand
    3. duplicity was never developed for data sets this huge. you may run into trouble.
    4. for big data sets a strategy to backup to a local file system and mirror that to a cloud location later might improve performance a lot.