Guide for rolling out breaking changes

Report an issue View source Nightly · 7.4 . 7.3 · 7.2 · 7.1 · 7.0 · 6.5

It is inevitable that we will make breaking changes to Bazel. We will have to change our designs and fix the things that do not quite work. However, we need to make sure that community and Bazel ecosystem can follow along. To that end, Bazel project has adopted a backward compatibility policy. This document describes the process for Bazel contributors to make a breaking change in Bazel to adhere to this policy.

  1. Follow the design document policy.

  2. File a GitHub issue.

  3. Implement the change.

  4. Update labels.

  5. Update repositories.

  6. Flip the incompatible flag.

GitHub issue

File a GitHub issue in the Bazel repository. See example.

We recommend that:

  • The title starts with the name of the flag (the flag name will start with incompatible_).

  • You add the label incompatible-change.

  • The description contains a description of the change and a link to relevant design documents.

  • The description contains a migration recipe, to explain users how they should update their code. Ideally, when the change is mechanical, include a link to a migration tool.

  • The description includes an example of the error message users will get if they don't migrate. This will make the GitHub issue more discoverable from search engines. Make sure that the error message is helpful and actionable. When possible, the error message should include the name of the incompatible flag.

For the migration tool, consider contributing to Buildifier. It is able to apply automated fixes to BUILD, WORKSPACE, and .bzl files. It may also report warnings.

Implementation

Create a new flag in Bazel. The default value must be false. The help text should contain the URL of the GitHub issue. As the flag name starts with incompatible_, it needs metadata tags:

      metadataTags = {
        OptionMetadataTag.INCOMPATIBLE_CHANGE,
      },

In the commit description, add a brief summary of the flag. Also add RELNOTES: in the following form: RELNOTES: --incompatible_name_of_flag has been added. See #xyz for details

The commit should also update the relevant documentation, so that there is no window of commits in which the code is inconsistent with the docs. Since our documentation is versioned, changes to the docs will not be inadvertently released prematurely.

Labels

Once the commit is merged and the incompatible change is ready to be adopted, add the label migration-ready to the GitHub issue.

If a problem is found with the flag and users are not expected to migrate yet: remove the flags migration-ready.

If you plan to flip the flag in the next major release, add label `breaking-change-X.0" to the issue.

Updating repositories

Bazel CI tests a list of important projects at Bazel@HEAD + Downstream. Most of them are often dependencies of other Bazel projects, therefore it's important to migrate them to unblock the migration for the broader community. To monitor the migration status of those projects, you can use the bazelisk-plus-incompatible-flags pipeline. Check how this pipeline works here.

Our dev support team monitors the migration-ready label. Once you add this label to the GitHub issue, they will handle the following:

  1. Create a comment in the GitHub issue to track the list of failures and downstream projects that need to be migrated (see example)

  2. File Github issues to notify the owners of every downstream project broken by your incompatible change (see example)

  3. Follow up to make sure all issues are addressed before the target release date

Migrating projects in the downstream pipeline is NOT entirely the responsibility of the incompatible change author, but you can do the following to accelerate the migration and make life easier for both Bazel users and the Bazel Green Team.

  1. Send PRs to fix downstream projects.

  2. Reach out to the Bazel community for help on migration (e.g. Bazel Rules Authors SIG).

Flipping the flag

Before flipping the default value of the flag to true, please make sure that:

  • Core repositories in the ecosystem are migrated.

    On the bazelisk-plus-incompatible-flags pipeline, the flag should appear under The following flags didn't break any passing Bazel team owned/co-owned projects.

  • All issues in the checklist are marked as fixed/closed.

  • User concerns and questions have been resolved.

When the flag is ready to flip in Bazel, but blocked on internal migration at Google, please consider setting the flag value to false in the internal blazerc file to unblock the flag flip. By doing this, we can ensure Bazel users depend on the new behaviour by default as early as possible.

When changing the flag default to true, please:

  • Use RELNOTES[INC] in the commit description, with the following format: RELNOTES[INC]: --incompatible_name_of_flag is flipped to true. See #xyz for details You can include additional information in the rest of the commit description.
  • Use Fixes #xyz in the description, so that the GitHub issue gets closed when the commit is merged.
  • Review and update documentation if needed.
  • File a new issue #abc to track the removal of the flag.

Removing the flag

After the flag is flipped at HEAD, it should be removed from Bazel eventually. When you plan to remove the incompatible flag:

  • Consider leaving more time for users to migrate if it's a major incompatible change. Ideally, the flag should be available in at least one major release.
  • For the commit that removes the flag, use Fixes #abc in the description so that the GitHub issue gets closed when the commit is merged.