How to keep track of a table data-change migration on Ruby on Rails

Question

Here is the problem: I have Ruby on Rails project that has a table that have almost 100k rows and I have a binary column and I want to make changes in the content data on this column.

So I must iterate over those 100k rows, making changes on that row on particular column, saving it back on database.

But I must keep track of changes because these changes could fail and I should have someway to re-start data change from where I stopped.

Here is what I thought of a way of doing it:

• Create a Migration to have a table MigrationTrack to track all records that have being migrated
• Create a model of the above migration
• Create a rake task that grabs all 100k from TableToUpdate and iterate over them, saving data back to row and save its ID on MigrationTrack. Create a logic to have a join on TableToUpdate and MigrationTrack to filter only ids that I haven't updated yet
• After above migration finished create another migration to drop MigrationTrack table and remove its model.

Is there any other "Railsh way" to do that? Anyone have done such change?

Thanks

Answer 1

I would do it like this:

1. Add and deploy a migration adding a new column with the desired data type to the database table.
2. Add code to your model that save the value from the old column into the new column too.
3. Run a rake task or a simple one-liner in the console that touches all records to make sure the code introduced in step one ran on each record.

After this step, you can manually verify if all records in the database have both columns set as expected.

4. Switch using the new attribute instead of the old attribute in the code.
5. Drop the old column.