Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://realpython.com/move-django-model/ below:

How to Move a Django Model to Another App – Real Python

If you’ve ever thought about refactoring your Django app, then you might have found yourself needing to move a Django model around. There are several ways to move a Django model from one app to another using Django migrations, but unfortunately none of them are straightforward.

Moving models between Django apps is usually a very complicated task that involves copying data, changing constraints, and renaming objects. Because of these complications, the Django object-relational mapper (ORM) does not provide built-in migration operations that can detect and automate the entire process. Instead, the ORM provides a set of low-level migration operations that allow Django developers to implement the process themselves in the migration framework.

In this tutorial, you’ll learn:

• How to move a Django model from one app to another
• How to use advanced features of the Django migration command line interface (CLI), such as sqlmigrate, showmigrations, and sqlsequencereset
• How to produce and inspect a migration plan
• How to make a migration reversible and how to reverse migrations
• What introspection is and how Django uses it in migrations

After completing this tutorial, you’ll be able to choose the best approach for moving a Django model from one app to another based on your specific use case.

Free Bonus: Click here to get access to a free Django Learning Resources Guide (PDF) that shows you tips and tricks as well as common pitfalls to avoid when building Python + Django web applications.

Throughout this tutorial, you’re going work on a store app. Your store is going to start with two Django apps:

1. catalog: This app is for storing data on products and product categories.
2. sale: This app is for recording and tracking product sales.

After you finish setting up these two apps, you’re going to move a Django model called Product to a new app called product. In the process, you’ll face the following challenges:

• The model being moved has foreign key relationships with other models.
• Other models have foreign key relationships with the model being moved.
• The model being moved has an index on one of the fields (besides the primary key).

These challenges are inspired by real-life refactoring processes. After you overcome them, you’ll be ready to plan a similar migration process for your specific use case.

Before you start moving things around, you need to set up the initial state of your project. This tutorial uses Django 3 running on Python 3.8, but you can use similar techniques in other versions.

First, create your virtual environment in a new directory:

For step-by-step instructions on creating a virtual environment, check out Python Virtual Environments: A Primer.

In your terminal, activate the virtual environment and install Django:

You’re now ready to create your Django project. Use django-admin startproject to create a project called django-move-model-experiment:

After running this command, you’ll see that Django created new files and directories. For more about how to start a new Django project, check out Starting a Django Project.

Now that you have a fresh Django project, create an app with your store’s product catalog:

Next, add the following models to the new catalog app:

You’ve successfully created Category and Product models in your catalog app. Now that you have a catalog, you want to start selling your products. Create another app for sales:

Add the following Sale model to the new sale app:

Notice that the Sale model references the Product model using a ForeignKey.

To complete the setup, generate migrations and apply them:

For more about Django migrations check out Django Migrations: A Primer. With your migrations in place, you’re now ready to create some sample data!

To make the migration scenario as realistic as possible, activate the Django shell from your terminal window:

Next, create the following objects:

You created two categories, 'Shoes' and 'Clothes'. Next, you added two products, 'Pants' and 'Shirt', to the 'Clothes' category and one product, 'Boots', to the 'Shoes' category.

Congratulations! You’ve completed the setup for the initial state of your project. In a real-life scenario, this is where you would start planning your refactoring. Each of the three approaches presented in this tutorial will start from this point.

To kick things off, you’re going to take the long road:

1. Create a new model
2. Copy the data to it
3. Drop the old table

This approach has some pitfalls that you should be aware of. You’ll explore them in detail in the following sections.

Start by creating a new product app. From your terminal, execute the following command:

After running this command, you’ll notice a new directory called product was added to the project.

To register the new app with your existing Django project, add it to the list of INSTALLED_APPS in Django’s settings.py:

Your new product app is now registered with Django. Next, create a Product model in the new product app. You can copy the code from the catalog app:

Now that you’ve defined the model, try to generate migrations for it:

The error says that Django found two models with the same reverse accessor for the field category. This is because there are two models named Product that reference the Category model, creating a conflict.

When you add foreign keys to your model, Django creates a reverse accessor in the related model. In this case, the reverse accessor is products. The reverse accessor lets you access related objects like this: category.products.

The new model is the one you want to keep, so to resolve this conflict, remove the reverse accessor from the old model in catalog/models.py:

The attribute related_name can be used to explicitly set a related name for a reverse accessor. Here, you use the special value +, which instructs Django not to create a reverse accessor.

Now generate a migration for the catalog app:

Don’t apply this migration yet! Once this change takes place, code that used the reverse accessor might break.

Now that there’s no conflict between the reverse accessors, try to generate the migrations for the new product app:

Great! You’re ready to move on to the next step.

In the previous step, you created a new product app with a Product model that is identical to the model you want to move. The next step is to move the data from the old model to the new model.

To create a data migration, execute the following command from your terminal:

Edit the new migration file, and add the operation to copy the data from the old table:

To execute SQL in a migration, you use the special RunSQL migration command. The first argument is the SQL to apply. You also provide an action to reverse the migration using the reverse_sql argument.

Reversing a migration can come in handy when you discover a mistake and you want to roll back the change. Most built-in migration actions can be reversed. For example, the reverse action for adding a field is removing the field. The reverse action for creating a new table is dropping the table. It’s usually best to provide reverse_SQL to RunSQL so you can backtrack if something goes awry.

In this case, the forward migration operation inserts data from product_product into catalog_product. The backward operation will do the exact opposite, inserting data from catalog_product into product_product. By providing Django with the reverse operation, you’ll be able to reverse the migration in case of disaster.

At this point, you’re still halfway through the migration process. But there’s a lesson to be learned here, so go ahead and apply the migrations:

Before you move on to the next step, try to create a new product:

When you use an auto-incrementing primary key, Django creates a sequence in the database to assign unique identifiers to new objects. Notice, for example, that you didn’t provide an ID for the new product. You normally wouldn’t want to provide an ID because you want the database to assign primary keys for you using a sequence. However, in this case, the new table gave the new product the ID 1 even though this ID already existed in the table.

So, what went wrong? When you copied the data to the new table, you didn’t sync the sequence. To sync the sequence, you can use another Django admin command called sqlsequencereset. The command produces a script to set the current value of the sequence based on existing data in the table. This command is often used for populating new models with preexisting data.

Use sqlsequencereset to produce a script to sync the sequence:

The script generated by the command is database specific. In this case, the database is PostgreSQL. The script sets the current value of the sequence to the next value that the sequence should yield, which is the maximum ID in the table plus one.

To finish up, add the snippet to the data migration:

The snippet will sync the sequence when you apply the migration, solving the sequence issue you ran into above.

This detour to learn about syncing sequences has created a little mess in your code. To clean it up, delete the data in the new model from the Django shell:

Now that the data you copied is deleted, you can reverse the migration. To reverse a migration, you migrate to a previous migration:

You first used the command showmigrations to list the migrations applied to the app product. The output shows that both migrations were applied. You then reversed migration 0002_auto_20200124_1300 by migrating to the prior migration, 0001_initial.

If you execute showmigrations again, then you’ll see that the second migration is no longer marked as applied:

The empty box confirms that the second migration has been reversed. Now that you have a clean slate, run the migrations with the new code:

The migration was successfully applied. Make sure you can now create a new Product in the Django shell:

Amazing! Your hard work paid off, and you’re ready for the next step.

The old table currently has other tables referencing it using a ForeignKey field. Before you can remove the old model, you need to change the models referencing the old model so that they reference the new model.

One model that still references the old model is Sale in the sale app. Change the foreign key in the Sale model to reference the new Product model:

Generate the migration and apply it:

The Sale model now references the new Product model in the product app. Because you already copied all the data to the new model, there are no constraint violations.

The previous step eliminated all references to the old Product model. It’s now safe to remove the old model from the catalog app:

Generate a migration but don’t apply it yet:

To make sure the old model is deleted only after the data is copied, add the following dependency:

Adding this dependency is extremely important. Skipping this step can have terrible consequences, including lost data. For more about migration files and dependencies between migrations, check out Digging Deeper Into Django Migrations.

Note: The name of the migration includes the date and the time it was generated. If you’re following along with your own code, then these parts of the name will be different.

Now that you’ve added the dependency, apply the migration:

The transfer is now complete! You’ve successfully moved the Product model from the catalog app to the new product app by creating a new model and coping the data to it.

One of the benefits of Django migrations is that they’re reversible. What does it mean for a migration to be reversible? If you make a mistake, then you can reverse the migration and the database will revert back to the state from before the migration was applied.

Remember how you previously provided reverse_sql to RunSQL? Well, this is where it pays off.

Apply all the migrations on a fresh database:

Now, reverse them all using the special keyword zero:

The database is now back to its original state. If you deploy this version and discover a mistake, then you can reverse it!

When you’re moving models from one app to another, some Django features may require special attention. In particular, adding or modifying database constraints and using generic relations both require extra care.

Adding constraints to tables with data can be a dangerous operation to perform on a live system. To add a constraint, the database must first verify it. During the verification, the database acquires a lock on the table, which might prevent other operations until the process completes.

Some constraints, such as NOT NULL and CHECK, may require a full scan of the table to verify that the new data is valid. Other constraints, such as FOREIGN KEY, require validation with another table, which can take some time depending on the size of the referenced table.

If you’re using generic relations, then you might need an extra step. Generic relations use both the primary key and the content type ID of the model to reference a row in any model table. The old model and the new model do not have the same content type ID, so generic connections can break. This can sometimes go unnoticed because the integrity of generic foreign keys is not enforced by the database.

There are two ways to deal with generic foreign keys:

1. Update the content type ID of the new model to that of the old model.
2. Update the content type ID of any referencing tables to that of the new model.

Either way you choose, make sure to test it properly before deploying to production.

Moving a Django model to another app by copying the data has its advantages and disadvantages. Here are some of the pros associated with this approach:

• It’s supported by the ORM: Performing this transition using built-in migration operations guarantees proper database support.
• It’s reversible: Reversing this migration is possible if necessary.

Here are some of the cons associated with this approach:

• It’s slow: Copying large amounts of data can take time.
• It requires downtime: Changing the data in the old table while it’s being copied to the new table will cause data loss during the transition. To prevent this from happening, downtime is necessary.
• It requires manual work to sync the database: Loading data to existing tables requires syncing sequences and generic foreign keys.

As you’ll see in the following sections, using this approach to move a Django model to another app takes much longer than other approaches.

In the previous approach, you copied all the data to the new table. The migration required downtime, and it could take a long time to complete depending on how much data there is to copy.

What if, instead of copying the data, you changed the new model to reference the old table?

This time, you’re going to make all the changes to the models at once, and then let Django generate all the migrations.

First, remove the Product model from the catalog app:

You’ve removed the Product model from the catalog app. Now move the Product model to the new product app:

Now that the Product model exists in the product app, you can change any references to the old Product model to reference the new Product model. In this case, you’ll need to change the foreign key in sale to reference product.Product:

Before moving on to generating the migrations, you need to make one more small change to the new Product model:

Django models have a Meta option called db_table. With this option, you can provide a table name to use in place of the one generated by Django. This option is most commonly used when setting up the ORM on an existing database schema in which the table names don’t match Django’s naming convention.

In this case, you set the name of the table in the product app to reference the existing table in the catalog app.

To complete the setup, generate migrations:

Before you move forward, produce a migration plan using the --plan flag:

The output of the command lists the order in which Django is going to apply the migrations.

The main benefit of this approach is that you don’t actually make any changes to the database, only to the code. To eliminate changes to the database, you can use the special migration operation SeparateDatabaseAndState.

SeparateDatabaseAndState can be used to modify the actions Django performs during a migration. For more about how to use SeparateDatabaseAndState, check out How to Create an Index in Django Without Downtime.

If you look at the contents of the migrations Django generated, then you’ll see that Django creates a new model and deletes the old one. If you execute these migrations, then the data will be lost, and the table will be created empty. To avoid this, you need to make sure Django does not make any changes to the database during the migration.

You can eliminate changes to the database by wrapping each migration operation in a SeparateDatabaseAndState operation. To tell Django not to apply any changes to the database, you can set db_operations to an empty list.

You plan on reusing the old table, so you need to prevent Django from dropping it. Before dropping a model, Django will drop the fields referencing the model. So, first, prevent Django from dropping the foreign key from sale to product:

Now that Django has handled the related objects, it can drop the model. You want to keep the Product table, so prevent Django from dropping it:

You used database_operations=[] to prevent Django from dropping the table. Next, prevent Django from creating the new table:

Here, you used database_operations=[] to prevent Django from creating a new table. Finally, you want to prevent Django from re-creating the foreign key constraint from Sale to the new Product model. Since you’re reusing the old table, the constraint remains in place:

Now that you’re done editing the migration files, apply the migrations:

At this point, your new model points to the old table. Django didn’t make any changes to the database, and all the changes were made to Django’s model state in the code. But before you call this a success and move forward, it’s worth confirming that the new model’s state matches that of the database.

To make sure the state of the models is consistent with the state of the database, try to make a change to the new model, and make sure Django detects it correctly.

The Product model defines an index on the name field. Remove that index:

You removed the index by eliminating db_index=True. Next, generate a migration:

Before moving forward, inspect the SQL generated by Django for this migration:

Great! Django detected the old index, as shown by the "catalog_*" prefix. Now you can execute the migration:

Make sure you got the expected result in the database:

Success! The index on the name column was dropped.

Changing the model to reference another model has its advantages and disadvantages. Here are some of the pros associated with this approach:

• It’s fast: This approach doesn’t make any changes to the database, so it’s very fast.
• It doesn’t require downtime: This approach doesn’t require copying data, so it can be performed on a live system without downtime.
• It’s reversible: It’s possible to reverse this migration if necessary.
• It’s supported by the ORM: Performing this transition using built-in migration operations guarantees proper database support.
• It doesn’t require any sync to the database: With this approach, related objects, such as indices and sequences, remain unchanged.

The only major con to this approach is that it breaks the naming convention. Using the existing table means the table will still use the name of the old app.

Notice how this approach is much more straightforward than copying the data.

In the previous example, you made the new model reference the old table in the database. As a result, you broke the naming convention used by Django. In this approach, you do the reverse: You make the old table reference the new model.

More specifically, you create the new model and generate a migration for it. You then take the name of the new table from the migration Django created and, instead of creating the table for the new model, you rename the old table to the name of the new table using the special migration operation AlterModelTable.

Just like before, you start by creating a new product app to make all the changes at once. First, remove the Product model from the catalog app:

You’ve eliminated Product from catalog. Next, move the Product model to a new product app:

The Product model now exists in your product app. Now change the foreign key in Sale to reference product.Product:

Next, let Django generate migrations for you:

You want to prevent Django from dropping the table because you intend to rename it.

To get the name of the Product model in the product app, generate the SQL for the migration that creates Product:

The name of the table that Django generated for the Product model in the product app is product_product.

Now that you have the name Django generated for the model, you’re ready to rename the old table. To drop the Product model from the catalog app, Django created two migrations:

1. catalog/migrations/0002_remove_product_category removes the foreign key from the table.
2. catalog/migrations/0003_delete_product drops the model.

Before you can rename the table, you want to prevent Django from dropping the foreign key to Category:

Using SeparateDatabaseAndState with database_operations set to an empty list prevents Django from dropping the column.

Django provides a special migration operation, AlterModelTable, to rename a table for a model. Edit the migration that drops the old table, and instead rename the table to product_product:

You used SeparateDatabaseAndState along with AlterModelTable to provide Django with a different migration action to execute in the database.

Next, you need to prevent Django from creating a table for the new Product model. Instead, you want it to use the table you renamed. Make the following change to the initial migration in the product app:

The migration creates the model in Django’s state, but it doesn’t create the table in the database because of the line database_operations=[]. Remember when you renamed the old table to product_product? By renaming the old table to the name Django would have generated for the new model, you force Django to use the old table.

Finally, you want to prevent Django from re-creating the foreign key constraint in the Sale model:

You’re now ready to run the migration:

Great! The migration was successful. But before you move on, make sure it can be reversed:

Amazing! The migration is completely reversible.

Note: AlterModelTable is generally preferable to RunSQL for a few reasons.

First, AlterModelTable can handle many-to-many relations between fields whose names are based on the model’s name. Using RunSQL to rename the tables may require some additional work.

Also, built-in migration operations such as AlterModelTable are database agnostic, whereas RunSQL is not. If, for example, your app needs to work on multiple database engines, then you might run into some trouble writing SQL that’s compatible with all database engines.

The Django ORM is an abstraction layer that translates Python types to database tables and vice versa. For example, when you created the model Product in the product app, Django created a table called product_product. Besides tables, the ORM creates other database objects, such as indices, constraints, sequences, and more. Django has a naming convention for all of these objects based on the names of the app and model.

To get a better sense of what it looks like, inspect the table catalog_category in the database:

The table was generated by Django for the Category model in the app catalog, hence the name catalog_category. You can also notice a similar naming convention for the other database objects.

• catalog_category_pkey refers to a primary key index.
• catalog_category_id_seq refers to a sequence to generate values for the primary key field id.

Next, inspect the table of the Product model you moved from catalog to product:

At first glance, there are more related objects. A closer look, however, reveals that the names of the related objects are not consistent with the name of the table. For example, the name of the table is product_product, but the name of the primary key constraint is catalog_product_pkey. You copied the model from the app called catalog, so that must mean that the migration operation AlterModelTable does not change the names of all related database objects.

To better understand how AlterModelTable works, inspect the SQL generated by this migration operation:

What this shows is that AlterModelTable only renames tables. If that’s the case, then what would happen if you tried to make a change to one of the database objects related to the tables of these objects? Would Django be able to handle these changes?

To find out, try to remove the index on the field name in the Product model:

Next, generate a migration:

The command succeeded—that’s a good sign. Now inspect the generated SQL:

The generated SQL commands drop the index catalog_product_name_924af5bc. Django was able to detect the existing index even though it’s not consistent with the name of the table. This is known as introspection.

Introspection is used internally by the ORM, so you won’t find much documentation about it. Each database backend contains an introspection module that can identify database objects based on their properties. The introspection module would usually use the metadata tables provided by the database. Using introspection, the ORM can manipulate objects without relying on the naming convention. This is how Django was able to detect the name of the index to drop.

Renaming the table has its advantages and disadvantages. Here are some of the pros associated with this approach:

• It’s fast: This approach only renames database objects, so it’s very fast.
• It doesn’t require downtime: With this approach, database objects are locked only for a short time while they are renamed, so it can be performed on a live system without downtime.
• It’s reversible: Reversing this migration is possible if necessary.
• It’s supported by the ORM: Performing this transition using built-in migration operations guarantees proper database support.

The only potential con associated with this approach is that it breaks the naming convention. Renaming only the tables means that the names of other database objects will be inconsistent with Django’s naming convention. This may cause some confusion when working with the database directly. However, Django can still use introspection to identify and manage these objects, so this is not a major concern.

In this tutorial, you’ve learned how to move a Django model from one app to another in three different ways. Here’s a comparison of the approaches described in this tutorial:

Note: The table above suggests that renaming the table preserves Django’s naming convention. While this is not strictly true, you learned earlier that Django can use introspection to overcome the naming issues associated with this approach.

Each of the approaches above has its own advantages and disadvantages. So, which approach should you use?

As a general rule of thumb, you should copy the data when you’re working with a small table and you can afford some downtime. Otherwise, your best bet is to rename the table and reference the new model to it.

That said, every project has its own unique requirements. You should choose whichever approach makes the most sense for you and your team.

After reading this tutorial, you’re in a better position to make the right decision about how to move a Django model to another app based on your specific use case, restrictions, and requirements.

In this tutorial, you’ve learned:

• How to move a Django model from one app to another
• How to use advanced features of the Django migration CLI, such as sqlmigrate, showmigrations, and sqlsequencereset
• How to produce and inspect a migration plan
• How to make a migration reversible, and how to reverse migrations
• What introspection is and how Django uses it in migrations

To dive deeper, check out the full range of database tutorials and Django tutorials.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4