WIP Rebuild migrations

[michaeljcollinsuk]

UPDATE - see latest comment below about how to run this locally. Leaving the old commits and description for now though as an example of the alternative

This adds a bash script to reset migrations and recreate them. As this will need to be run on the server manually it would be good to have this tested by others first. Before running, I recommend taking a backup of your database in case anything goes wrong.

The script will do the following:

• Clear the migration history for all our apps by using the --fake flag with migrate to reset the migrations table in the database
• Then delete all existing migration files
• Recreate them using makemigrations. This creates a set of new initial files which match our current schema
• Fake apply these migration files again using the --fake flag

This will result in uncommitted changes, because we need to keep the old migration files in place to un-apply them before we can generate the new ones. This means there is a slightly awkward scenario of needing to run this on the server and then commit the changes there, where they will need to be pushed back to the repo - but I'm not sure if this is possible?

To avoid this, I could remove the find and delete command from the script, and run this locally and commit it separately. Then on the server we could checkout the previous commit to run the script to reset the migrations, and then checkout the later commit to apply the new files.

[michaeljcollinsuk]

Having opened this, I think it is actually better to do this in two steps that then dont rely on committing any changes on the server, as per last comment. So now the process to run this (and test locally) would be to:

• Take a DB backup e.g:
  pg_dump -Fc ynr > ynr_local_backup.dump
• Checkout commit fc2c885 - this contains our old migration files:
  git checkout fc2c88528dd87cb5a4da501410ab4abfbfce6406
• Run the reset_migrations.sh script to reset the migration history:
  ./reset_migrations.sh
• Checkout head of this branch - this contains the new squashed migrations:
  git checkout squash-migrations
• Fake apply the new migration files without actually altering our schema:
  python manage.py migrate --fake

Another thought I had was that we may want to do this when we dont think anyone will be using the site, or even disable access temporarily by stopping gunicorn? In case there is an issue with a DB record being updated at the same time as we are running these commands. Although as the schema will not actually alter perhaps its not an issue, I'm not sure.

[symroe]

Couple of quick fire questions:

1. Can you explain more about what 72c6a69 is doing in the commit message? I can't see why these changes are needed
2. Is there reason to use this method and not squashmigrations? IIRC, that will create a new migration with a replaces list of previous migrations, and I think Django manages the database table itself? I also think that allows us to check in the new migration file and remove the old ones in a single commit, but I'd never done it so I'd like to hear your thougths

[michaeljcollinsuk]

Couple of quick fire questions:

1. Can you explain more about what 72c6a69 is doing in the commit message? I can't see why these changes are needed
2. Is there reason to use this method and not squashmigrations? IIRC, that will create a new migration with a replaces list of previous migrations, and I think Django manages the database table itself? I also think that allows us to check in the new migration file and remove the old ones in a single commit, but I'd never done it so I'd like to hear your thougths

I've updated the commit message to explain why I made those changes in the migration file. Also this blog has more detail about the approach in general