Django MIGRATION_MODULES

There's not a lot about MIGRATION_MODULES in the Django project documentation:

MIGRATION_MODULES Settings | Django documentation | Django (djangoproject.com)

By default it's an empty dictionary, which can be modified to specify for each app, where the migration modules can be found and what you'd like to name directory for your migrations. The key is the app name and the value is the location and file name: {[app_label]: [directory].[migration_file_name]}

Using puput, the Wagtail (wagtail docs) based blog engine for the Djangonaut Space blog, I ended up creating migration files where my package is in my virtual environment instead of with the rest of my project files. You can install puput as a standalone app, or you can add it to an existing Wagtail application. It's also what I use for the blog you're reading right now. I really like this package, I absolutely recommend it and this solution is mentioned in their docs.

It looked like this:

Pro Tip: Always specify your app(s) when you're running your migrations. Just like you should always git status and not indiscriminately git add . before you commit; specifying your apps when you make your migrations and run your migrations will make sure you didn't half finish a feature and it gets mixed up with that task at hand.

I usually:

1. git status to see all the models.py files I've touched, especially when I'm solo hacking and I have a tendency to just add features and forget what issue I'm working on.
2. Then python manage.py makemigrations . Although I can specify the app_label(s) as an argument here, I want to see where Django detects changes. (Read more: How Django Detects Changes - Digging Deeper Into Django Migrations – Real Python).
3. I migrate specifying the app_label and the migration name with the command python manage.py migrate [app_label] [migration_name]
4. Then you do some user testing, then run your Playwright tests and if everything is good to go then you squash_migrations , commit and push your code.

After talking this over at DjangoCon US 2023, this happens more than just with puput. I don't know all of the packages it could happen with, but if you are creating your own package with its own models.py without package settings to handle this problem in a different way, this could happen to you.

I have a half baked idea to solve this problem programmatically. I wonder if something like this could work:

This solution would produce a file structure like this:

In my case, I already had the application deployed with a half done blog (oops 😅).

One option is to dump my postgresql database in JSON, JSONL, XML or YAML (the serializer formats Django supports), edit the JSON (!? bad? but it'd be with a Python script and I've done riskier things) and then do something like python manage.py loaddata fixture/whole.json with a fixture and then running tests to make sure everything is in the correct fields.

That feels a bit risky. If you've ever poked around the tables Wagtail creates, you'll see it's nontrivial.

Thankfully it wasn't a working blog with real entries I need to keep track of so I'll use my favorite option: destroy it and start over💥. On my production server (I'm testing it out on staging server first) , I zero my migrations for the puput application with the command python manage.py migrate puput zero , then deleted all of the migration files and the migrations directory. Locally I've done the same actions: zero out migrations so they are unapplied for that app and then delete the migrations directory in virtual environment. I added MIGRATION_MODULES = {"puput": "home.puput_migrations"} to my settings/base.py . Then ran python manage.py makemigrations puput locally. I'll commit these migrations files. When I deploy the code to my production server and run python manage.py migrate it will start my puput blog from scratch.

The results of local_djangonaut_space# \dt 🙀

public | wagtailadmin_admin | table | postgres

public | wagtailcore_collection | table | postgres

public | wagtailcore_collectionviewrestriction | table | postgres

public | wagtailcore_collectionviewrestriction_groups | table | postgres

public | wagtailcore_comment | table | postgres

public | wagtailcore_commentreply | table | postgres

public | wagtailcore_groupapprovaltask | table | postgres

public | wagtailcore_groupapprovaltask_groups | table | postgres

public | wagtailcore_groupcollectionpermission | table | postgres

public | wagtailcore_grouppagepermission | table | postgres

public | wagtailcore_locale | table | postgres

public | wagtailcore_modellogentry | table | postgres

public | wagtailcore_page | table | postgres

public | wagtailcore_pagelogentry | table | postgres

public | wagtailcore_pagesubscription | table | postgres

public | wagtailcore_pageviewrestriction | table | postgres

public | wagtailcore_pageviewrestriction_groups | table | postgres

public | wagtailcore_referenceindex | table | postgres

public | wagtailcore_revision | table | postgres

public | wagtailcore_site | table | postgres

public | wagtailcore_task | table | postgres

public | wagtailcore_taskstate | table | postgres

public | wagtailcore_workflow | table | postgres

public | wagtailcore_workflowpage | table | postgres

public | wagtailcore_workflowstate | table | postgres

public | wagtailcore_workflowtask | table | postgres

public | wagtaildocs_document | table | postgres

public | wagtaildocs_uploadeddocument | table | postgres

public | wagtailembeds_embed | table | postgres

public | wagtailforms_formsubmission | table | postgres

public | wagtailimages_image | table | postgres

public | wagtailimages_rendition | table | postgres

public | wagtailimages_uploadedimage | table | postgres

public | wagtailredirects_redirect | table | postgres

public | wagtailsearch_indexentry | table | postgres

public | wagtailsearch_query | table | postgres

public | wagtailsearch_querydailyhits | table | postgres

public | wagtailusers_userprofile | table | postgres

Pro Tip: For Wagtail, the StreamField or StreamBlock will create a new migration for every change, even though it's just a JSON object. That took me a bit of poking around! Thanks Vince Salvino @ Code Red!

The idea and bug inspiring this blog post came from the DjangoCon US 2023 Sprints!