Complex data migrations with South

You’re in the middle of a project when you realise that your database model needs a refactoring to support latest changes. What to do? Meet South, a migration tool for Django applications. It will track changes in your models and help you migrate as your project develops. Most changes can be detected automatically, like adding and removing fields. Once you start renaming things and moving them it can get tricky.

WARNING: always make backups before doing any migrations

Let’s say we have a Todo application with these models:

But you later on figure out that it would actually be much better if you’d refactor to this:

Now, if you just change the models to this and run auto migration, South will remove all your data and create new models. Not really what you expected.

As South can only detect simple changes, we need to be smart, so what to do?

Lets first generate new models by appending them in our models.py, so we can use south normally.

This will detect new models and generate them. Sweet, but what about data? Meet data migrations.

./manage.py datamigration todolist move_to_submodels

This will create an empty 00NN_move_to_submodels.py file, ready to be used for data migration. Lets say that Todo instances with location or all_day field set are Events otherwise it’s a Task.

We add this code:

We now migrate again and check if data was migrated correctly. Once you’re sure everything is ok, you can delete Todo model from models.py and migrate to delete the table forever.

That’s it.