How Allegro Does Automated Code Migrations for over 2000 Microservices

Write for InfoQ

Allegro shared the details of the process it uses to manage code migrations at scale. The company combined GitHub’s Dependabot and OpenRewrite projects into a custom solution that helps developers perform mundane code migration tasks automatically across numerous source code repositories. The company tackled many edge cases to ensure the process operates smoothly, relieving initial trust issues among developers.

Allegro created and manages a large microservices platform, spanning over 2000 services. At such a scale, code migrations can become a headache. Bartosz Gałek, principal engineer at Allegro, describes the challenges with applying code migrations manually:

Migrating code across numerous repositories is incredibly painful when new versions of a company-wide library introduce breaking changes. Traditionally, developers must follow migration guides in release notes, identify the required changes, and manually update their code. This process is not only time-consuming but also prone to human error, which can lead to inconsistencies and potential issues in production. It also raises many questions on Slack support channels.

The company opted to automate as many code migrations as possible and settled on using GitHub's Dependabot and OpenRewrite project. Engineers created a custom GitHub application named allegro-rewrite and set themselves a mission to provide a developer-friendly process for supporting automated code migrations at scale. The team behind the project wanted to achieve specific goals, such as supporting easy adoption of new tools, providing auditability and reversibility, implementing a deadline process for time-sensitive migrations, and ensuring the migration can be easily re-run if needed.

The solution relies on Dependabot continuously monitoring Allegro's GitHub organization for outdated dependencies. When Dependabot creates a pull request, the custom allegro-rewrite GitHub application triggers a workflow, which applies a series of OpenRewrite recipes to address breaking changes, based on the change use case.

Automated Code Migration Flow (Source: Allegro Technology Blog)

The team extended the initial implementation in many ways. Engineers introduced comment commands for GitHub pull requests, enabling interactions between PR reviewers and the allegro-rewrite application. Additionally, the application supports force-merging time-sensitive changes, such as when security vulnerabilities affecting many repositories must be addressed by a specific deadline. They also made the same application available as a CLI one so that the company’s engineers can use it locally and potentially leverage it for other use cases, beyond GitHub-related workflows.

Using CLI to run a custom OpenRewrite recipe (Source: Allegro Technology Blog)

Rolling out the automated process for code migrations faced some challenges. Initially, the company's developers were quite sceptical about performing massive migrations in an automated fashion, worrying about the number of changes applied and the possibility of issues being introduced at scale. Furthermore, unforeseen post-deployment issues were encountered with code parsing and YAML formatting on a few occasions. Still, the team swiftly improved recipe test coverage to avoid such problems in the future. Lastly, the automated migration process resulted in an extra effort for internal library maintainers if they needed to implement OpenRewrite recipes to help deal with breaking changes introduced in new library versions.

Allegro developers are still working on further improvements and plan to open-source the solution so that other organizations can benefit from their experiences.

Mark Brown commented on the blog post:

This is amazing. We were just attempting to solve the same problem. Looking forward to being able to use this tool. Nice job, guys!

Related Topics:

• Development
• Architecture & Design
• Java
• Spring Boot
• migration
• Automation
• Refactoring
• Microservices
• Continuous Integration
• github
• Continuous Delivery
• Developer Experience

We protect your privacy.