eZ migrations made easy - Kaliop migration bundle

Offering a helping hand, Kaliop implemented a migration bundle - a tool that helps migrate changes you made in eZ administration to any other environment. It is inspired by the DoctrineMigrationsBundle and is fairly similar. It is used through the command line: on the instance you made changes to, you should generate the migration file executing one simple command and then, after deploying that file to another instance, apply that migration executing an even simpler command.

Let me try to demonstrate how we used this bundle and how it helped my team and me on a few projects. We were using this version, but last year ezmigrationbundle2 was created to support eZ Platform 3 so you can also check that out. This post will be based on the first version, but all the commands and information should be the same for the updated second version. 

Migration Bundle Workflow

The first step is installation which is fairly straightforward. Once you have Kaliop migration bundle up and running, I suggest checking the help message of the generate command:

$ kaliop:migration:generate --help

Which will provide valuable info on arguments and options of the given command.

To provide an example, let’s say you are working in the AppBundle bundle and you have an existing content type with identifier ng_article to which you just added a new field definition. If you’d like to generate a migration file for this change, you would execute:

$ kaliop:migration:generate --type=content_type --match-type=content_type_identifier --match-value=ng_article --mode=update AppBundle

--type flag here suggests what we want to migrate: it can be content, content versions, content type groups, languages, roles, tags and more. 

--match-type and --match-value go together and point the bundle to the exact objects we want to migrate. Specifically, type tells what property to look at to identify those entities (for example, content_type_identifier) and value is the value of that property in our entities.

--mode-update the migration mode (create, update, delete) - by default is set to create

After executing the upper command, you should be able to find a file named with the following pattern: YYYYMMDDHHMMSS_placeholder.yml. The advice is to replace the placeholder in the name with something more significant to provide information to other developers about what is the summary of migration. In our example, we could replace it with “ng_article_update”.

TIP: To save time, you can also pass the migration file name as a parameter to the migration:generate command, after the bundleName. That would mean the last command would be

$ kaliop:migration:generate --type=content_type --match-type=content_type_identifier --match-value=ng_article --mode=update AppBundle ng_article_update

And would create a file called YYYYMMDDHHMMSS_ng_article_update.yml.

Now when you have the migration file, you can easily perform the migration on any other instance: be it a colleague’s local instance for testing before deployment or a production server after deployment. Once they have the migration file(s) on their instance, they only need to execute:

$ kaliop:migration:migrate

After the successful execution, your content model for further development or content exploring should be the same.

Caution: no rollback option

I mentioned earlier that this bundle is similar to Doctrine Migrations Bundle. Having said that, it is important to mention one big difference: Kaliop Migration Bundle offers no rolling back options after executing migrations. That’s why the official recommendation is to always take a database snapshot and then execute the migrations, to have a backup in case you want to roll back the changes.

The versatility of the migration generate command

Take note of the command 

$ kaliop:migration:generate --list-types

which lists all migration types which are available, along with their match types. You’ll find lots of types here, from the basic content and content type, to section and roles… Let me list a few examples of commands for generating migrations of various migration types, to get a grasp of the versatility of the migration:generate command.

Scenario 1: After setting up new languages in the administration, you want to migrate them all.

$ kaliop:migration:generate --type=language --match-type=language_code --match-value=fre-FR,ger-DE,ita-IT,pol-PL AppBundle

 Scenario 2: You had adjusted policies for an existing role Member, checked if everything works as intended and want to migrate this setup to other environments.

$ kaliop:migration:generate --type=role --match-type=role_identifier --match-value=Member --mode=update AppBundle

 Scenario 3: You added new tags for categorizing articles via the Netgen Tags Bundle and don’t want to manually have to add these to all environments.

$ kaliop:migration:generate --type=tag --match-type=tag_keyword --match-value=Beauty,Food,Sports AppBundle

Problems

Using this bundle, I’ve encountered a few situations when the migration file was not well generated and the migrate command caused errors for that file. The solution was always quite clear, with just a bit of debugging on what Kaliop does while generating the migration files. A command that helps when your migration files run into an error is

$ kaliop:migration:status

which gives a list of all migration files and their statuses, along with the error message if a migration file failed.

One thing to note here is that, if you find and fix the error that prevented a file from being executed, you must delete the info about that migration failing before trying to execute the migration again. Otherwise, you will just get a message saying there are no migrations to execute. To delete a migration file, run:

$ kaliop:migration:migration YYYYMMDDHHMMSS_placeholder.yml --delete

With the correct name of the file whose migration failed.

With all of the errors I’ve encountered, I’ve managed to find an easy work-around so let me list a few situations and their solutions, for a quick grasp of what you might encounter.

• Problem 1: Referencing a parent location

If you try to create a migration of some test data where one content is a sub-item of another, the migration file will connect the two content objects via parent_location parameter, assigning the child the location id of the parent.

parent_location: 24837

However, the location id of the wanted content might be different on another instance, so you might get an error of non-existent location_id, or even worse, your child content might end up under the wrong parent without you even noticing it. That’s why it is better to create a reference for the parent location and use that reference in the parent_location parameter.

To do this, add the following property to the parent content in the migration file:

references:
    -   identifier: products_container
        attribute: location_id

And then, in the child content properties, we’ll use like this: 

parent_location: reference :products_container

This way, the child will surely be located under the right parent.

• Problem 2: Content with non-empty custom object relation list field (only if using Netgen media-site)

If you are using Netgen media-site and you want to migrate some content that has a non-empty custom object relation list field (let the identifier be authors), generating the migration will produce a yml file with this syntax:

authors:
    destinationContentIds:
        - 18629
    destinationLocationIds:
        - 18613

Even if executing this migration may not throw errors, it may create wrong content, so beware here. What happened here is that Kaliop Migration Bundle will handle both these given values as content ids. In the given example, two objects will be set as related to our new content: one with the content id 18629 and the other with the id 18613. To prevent this, just modify the yml file by deleting destiantionLocationIds and its values:

authors:
    destinationContentIds:
        - 18629

This should now create content as expected. If you need to place this content under a specific location, you can do that manually.

Conclusion

This was a quick introduction to the Kaliop migration bundle, its usage and commands, also pointing out a few examples of problems that might occur. Despite the problems, which are also constantly getting fewer as the Kaliop community tries to solve them, I think this bundle is a great tool for automating your initial setting of eZ structures and creating consistent content through your multiple environments.