Run breaking change detection

More information

Overview

We can detect breaking changes between versions of our repository. Buf is able to detect one of the following categories of breaking changes:

  • FILE: Generated source code breaking changes on a per-file basis, that is changes that would break the generated stubs where definitions cannot be moved across files. This makes sure that for languages such as C++ and Python where header files are included, your source code will never break for a given Protobuf change. This category also verifies wire and JSON compatibility.
  • PACKAGE: Generated source code breaking changes on a per-package basis, that is changes that would break the generated stubs, but only accounting for package-level changes. This is useful for languages such as Java (with option java_multiple_files = true; set) or Golang where it is fine to move Protobuf types across files, as long as they stay within the same Protobuf package. This category also verifies wire and JSON compatibility.
  • WIRE: Wire breaking changes, that is changes that would break wire compatibility, including checks to make sure you reserve deleted types of which re-use in the future could cause wire incompatibilities.
  • WIRE_JSON: Wire breaking changes and JSON breaking changes, that is changes that would break either wire compatibility or JSON compatibility. This mostly extends WIRE to include field and enum value names.

The default is FILE, which is our recommendation to guarantee maximum compatibility across your users. As opposed to linting, you generally will not mix and exclude specific breaking change checkers, instead choosing one of these options:

  • FILE
  • PACKAGE
  • WIRE
  • WIRE_JSON

We currently have the last option configured:

version: v1beta1
lint:
use:
- BASIC
- FILE_LOWER_SNAKE_CASE
except:
- ENUM_NO_ALLOW_ALIAS
- IMPORT_NO_PUBLIC
- PACKAGE_DIRECTORY_MATCH
- PACKAGE_SAME_DIRECTORY
- PACKAGE_AFFINITY
breaking:
use:
- WIRE_JSON

Create a base Image of your current state

Buf can check against any Input. For the sake of this tour, we will examine Image, git repository, tar, and zip inputs.

First, build an image of your current state to a file:

$ buf build -o image.bin

Make a wire-incompatible change

Let's change the type of a field within the file google/type/date.proto:

diff --git a/google/type/date.proto b/google/type/date.proto
index b958feeba..916f1da93 100644
--- a/google/type/date.proto
+++ b/google/type/date.proto
@@ -46,5 +46,5 @@ message Date {
// Day of month. Must be from 1 to 31 and valid for the year and month, or 0
// if specifying a year by itself or a year and month where the day is not
// significant.
- int32 day = 3;
+ string day = 3;
}

Run breaking change detection

$ buf check breaking --against image.bin
google/type/date.proto:49:3:Field "3" on message "Date" changed type from "int32" to "string".

Behind the scenes, this command:

  • Discovers all Protobuf files per your configuration.
  • Copies all Protobuf file content into memory.
  • Compiles all Protobuf files.
  • Parses the input image.
  • Compares the compilation result and input image for breaking changes.

We can also run against a committed git branch:

$ buf check breaking --against .git#branch=master
google/type/date.proto:49:3:Field "3" on message "Date" changed type from "int32" to "string".

Behind the scenes, this command:

  • Discovers all Protobuf files per your configuration.
  • Copies all Protobuf file content into memory.
  • Compiles all Protobuf files.
  • Clones the head of the master branch of the git repository located at local directory .git into memory.
  • Compiles all Protobuf files on the master branch per the configuration on that branch. Note that the master branch has no buf.yaml configuration file, but since googleapis has a single root that is the root of the repository, no configuration file is needed for building.
  • Compares the compilation results for breaking changes.

We can also run against a git tag, however googleapis does not have releases or tags. For a repository with the tag v1.0.0:

$ buf check breaking --against .git#tag=v1.0.0

Note that buf check breaking is able to produce references to your current files even if a type is moved between files, so for example if we moved the Date message to another file, Buf would reference the location within this file instead. It will also attempt to use an enclosing type for deleted references - for example, if a field is deleted, Buf will reference the enclosing message if still present. and if a nested message is deleted, Buf will reference the enclosing message as well.

Any input type can be used for either side of the comparison. See the Input documentation for more details, however we recommend waiting to do this until the end of the tour.

For one more example, let's compare against the tar and zip archives of the commit from GitHub:

$ buf check breaking --against "https://github.com/googleapis/googleapis/archive/${GOOGLEAPIS_COMMIT}.tar.gz#strip_components=1"
google/type/date.proto:49:3:Field "3" on message "Date" changed type from "int32" to "string".
$ buf check breaking --against "https://github.com/googleapis/googleapis/archive/${GOOGLEAPIS_COMMIT}.zip#strip_components=1"
google/type/date.proto:49:3:Field "3" on message "Date" changed type from "int32" to "string".

For remote locations that require authentication, see HTTPS Authentication and SSH Authentication for more details.

We can also output this as JSON:

$ buf check breaking --against "https://github.com/googleapis/googleapis/archive/${GOOGLEAPIS_COMMIT}.tar.gz#strip_components=1" --error-format=json
{"path":"google/type/date.proto","start_line":49,"start_column":3,"end_line":49,"end_column":9,"type":"FIELD_SAME_TYPE","message":"Field \"3\" on message \"Date\" changed type from \"int32\" to \"string\"."}

Once done, revert the change to google/type/date.proto and remove the image:

$ git reset --hard "${GOOGLEAPIS_COMMIT}"
$ rm image.bin

Read an image from stdin

As like any other buf command, buf check breaking can read input from stdin. This is useful if, for example, you are downloading an image file from a private location. As a fun but useless example, let's build an Image of our current state and output to stdout, then compare against the input from stdin. This should always pass as it is comparing the current state to the current state:

$ buf build -o - | buf check breaking --against -