What Precautions to take when dealing with production data?

Question

I have been developing different java web based applications. As a junior-mid level developer I spent my time mostly dealing with development data only in which it is not that much of a big deal erasing data, adding test data and so on. But now, I am doing a side project using Ruby on Rails by my own and responsible for every development stages and data (i.e on dev, prod, and test). Even at this stage I was sometimes erasing prod data (which is on Heroku) and re populating it using "db:populate" that is provided by Rails framework. However, the system is going live after a month (to hear feedback from users) and I know I won't have the option of erasing data if something went wrong from now on.

Based on your experience, what precautions should I take when something is wrong with real production data?

Answer 1

1. Write a class that can check Aspect X of the data and figure out what data is bad.
2. Write a class that can fix the bad Aspect X data.
3. Write tests for the class that create bad data and assert that after running the fixer, the data is good.
4. Create a copy of your production database and a copy of your application pointed to the copy of your production database (on Heroku this is easy).
5. Run the fixer class on the copy of the production database, and then verify with the checker class that the data is good. Also verify that the data is good by hand. You can do this from a console session via heroku run console.
6. Run the fixer class on the live copy of the production database, again verifying that the data is good.

Sometimes, you can use a class which is also an ActiveRecord Migration. But this will not always work out, and sometimes you must simply use a normal class you write from scratch yourself.