Bazaar limits the file size that it can commit based on the available virtual memory (according to an open bug).
I would like to put a database (as mysqldump text file) under version control. The database is 3 GB and I am working on a server with 64GB of memory. I do not understand why this would be a problem. When I attempt to commit, I get the error reported in the bug:
bzr: ERROR: exceptions.OverflowError: requested number of bytes is more than a Python string can hold
Is there a way that I can get this file under bazaar version control?
My preference for bazaar is that I am familiar with it, but I plan to automate the dump and check in as a cron job so any suitable version control system will do.
Two options I have come up with so far, until a better solution appears. Currently, I keep a copy of each weekly dump backup, storage is not an issue at this point. Otherwise, I could keep the first dump, diff the original and new version of the dump, and version control that. This would keep a record of the changes, but it would not be possible to return to an earlier state. I am not comfortable with this unless there would be a straightforward way to revert.
mysqldump mydb > mydb_base.sql
touch mydb_diff
bzr add mydb_diff
bzr commit -m 'first commit'
then in the cron script
mysqldump mydb > mydb.sql
diff mydb_base.sql mydb.sql > mydb_diff
bzr commit -m "`date +%Y.%m.%d-%H.%M` mydb diff" mydb_diff
Although I love Bazaar, I would not recommend that you use it for this purpose, unless you definitely need to keep every version of your data indefinitely. If in stead you only need a fixed number of past versions (e.g. 10) or only for a fixed period (eg. 2 years), then I'd suggest that you use an incremental backup tool like rdiff-backup.