Designing Django's Migrations

Andrew Godwin (Django core developer, creator of South)

Release: Django 1.7 comes out late May

Migrations

• They're pretty good

South

• ...it was good for it's time
• Has some underlying problems

The Initial Plan

• Django
  • Schema backend
  • ORM hooks
• South 2
  • migration handling
  • user interface

The Revised Plan

• Django
  • Schema backend
  • orm hooks
  • migration handling
  • user interface
• South 2
  • backport for Django 1.4 - 1.6

Logically Separate

• SchemaEditor
  • Schema backend
  • ORM Hooks
• Migrations
  • Migration handling
  • User interface

Not moving south into Django! Instead, adding migrations to Django.

SchemaEditor

• ORM : Table :: SchemaEditor : Schema
• Abstracts schema operations across DBs
• Works in terms of Django fields/models
• Contains per-database workarounds

django.db.migrations

• Migration file reader/writer
• Migration commands
• Dependency resolver
• Autodetector
• Applied/unapplied tracking

New Migration Format

• declarative
• define operations

  class Migration(django.db.migrations.Something):
        operations = [
            something...
        ]
  

In-memory running

• creates models from migration sets
• autodetector diffs created from on-disk
• used to feed SchemaEditor / ORM

Database Peculiarities

PostgreSQL

• It's great! Use it.

MySQL

• No transactional DDL
• No CHECK constraints
• Conflates UNIQUE and INDEX

Oracle

• Different SQL syntax
• Picky about names
• Can't convert to/from TextField (LOB)

SQLite

• AAAAAAAAAAAAHHHHHHHHHHHHHHHHHHHHH
• Altering tables? Schema introspection? What?
• Only has three DDL statements... CREATE TABLE, DROP TABLE, ADD COLUMN
• Fantastic embedded database... not made for this!

Backwards Compatability

• Django generally very good at this
• Auto-applies first migration if tables exist
  • If they tables already exist, Django marks the migration done
• Ignores South-style migrations
  • Old South migrations are not forward-compatible

Lessons Learnt

• Explicit is better than implicit
  • South had lots of implicit stuff
  • New migrations list out migration operations explicitly
• Abstracting DBs is hard
  • Generally true for abstracting
• Composability rocks
  • It's simplified the code so much
  • Makes migration squashing much easier
• Feedback is vital
  • I'm just not mad enough to do nasty things to my code