Usage

We highly recommend completing the tour to get an overview of buf check breaking.

Add a buf.yaml

To get started, add a buf.yaml file to the root of your repository that contains your Protobuf definitions.

This represents a buf.yaml file that sets the defaults for the build and for the breaking checkers:

version: v1beta1
build:
roots:
- .
breaking:
use:
- FILE

Adjust your roots as necessary. This correlates to your --proto_paths.

See the breaking configuration and build configuration documentation for more details.

Decide how to manage your previous Protobuf schema state and run breaking change detection

Breaking change detection is run using buf check breaking --against INPUT.

Buf's breaking change detector works by comparing a previous version of your Protobuf schema to your current version. Buf considers your current schema to be the "input" and your previous schema to be the "against input". This is represented by the first CLI argument <input> and --against.

There are multiple ways to store and/or retrieve your previous schema version due to Buf's implementation of Inputs. Here are some common scenarios and how to deal with them.

Compare directly against a git branch or git tag

You can directly compare against the .proto files at the head of a branch, or a tag. See the Inputs documentation for details on git branches and git tags.

As an example, if you are currently in the root of your git repository, you will have a .git directory. To compare against your Protobuf schema as committed on the master branch:

$ buf check breaking --against '.git#branch=master'

This will:

  • Discover all Protobuf files per your configuration.
  • Copy all Protobuf files into memory.
  • Compile all Protobuf files.
  • Clone the head of the master branch of the git repository located at local directory .git into memory.
  • Compile all Protobuf files on the master branch per the configuration on that branch.
  • Compare the compilation results for breaking changes.

This is especially useful for local development. Note that many CI services like Travis CI do not do a full clone of your repo, instead cloning a certain number of commits (typically around 50) on the specific branch that is being tested. In this scenario, other branches will not be present in your clone within CI, so the above won't work. While you could work around this by disabling git clone and doing it manually, a better alternative is to give the remote path directly to buf to clone itself:

# assuming your repo is github.com/foo/bar
$ buf check breaking --against 'https://github.com/foo/bar.git'

Buf only clones the single commit at the head of this branch, so even for large repositories, this should be quick.

You can also compare against a git tag, for example v1.0.0:

$ buf check breaking --against '.git#tag=v1.0.0'

You can also compare against a subdirectory in your git repository, if your buf.yaml is not stored at the root of your repository. For example, if your buf.yaml is stored in the subdirectory proto:

$ buf check breaking --against '.git#tag=v1.0.0,subdir=proto'

For remote locations that require authentication, see HTTPS Authentication and SSH Authentication for more details.

Note that GitHub Actions allows you to pull and create a local head for another branch, making it possible to just use i.e. .git or .git#branch=master. See buf-example for an example.

Compare against an archive

You can compare against a tar or zip archive of your .proto files as well. This is especially useful for GitHub where tarballs and zip archives can be retrieved for any commit or branch.

# assuming your repo is github.com/foo/bar and COMMIT is a variable storing the commit
# to compare against
$ buf check breaking --against "https://github.com/foo/bar/archive/${COMMIT}.tar.gz#strip_components=1'
$ buf check breaking --against "https://github.com/foo/bar/archive/${COMMIT}.zip#strip_components=1'

Store as an Image file

One way to manage your state is to store your Protobuf schema as an Image. The rough workflow is as follows:

  • When you merge to your golden branch (usually master), store the Protobuf schema as an image file using buf build -o image.bin. This will update a file image.bin that stores the current state of your Protobuf schema on master.
  • When on a branch and within CI, compare against this file using buf check breaking --against image.bin. Buf will automatically derive that image.bin is a binary Image file and use this for comparison.

To run:

$ buf check breaking --against image.bin

This will:

  • Discover all Protobuf files per your configuration.
  • Copy all Protobuf files into memory.
  • Compile all Protobuf files.
  • Parse the input Image from image.bin.
  • Compare the compilation result and input Image for breaking changes.

Note that you can use --exclude-source-info to save space if you'd like, since Buf does not use the SourceCodeInfo from your previous schema, however we'd recommend not setting this in case you want to use the SourceCodeInfo in the future.

$ buf build -o image-bin --exclude-source-info

There are various permutations of this. For example, instead of storing the image.bin file directly in your git repository, you can push to a remote location like an s3 bucket, and pull from this s3 bucket for comparisons.

$ buf check breaking --against https://s3.yourcompany.com/path/to/image.bin

Sometimes, your local repository will only have a subset of the files that are within an Image. In this case, you only want buf to check against the files that are currently in your local repository:

$ buf check breaking --against https://s3.yourcompany.com/path/to/image.bin --limit-to-input-files

Store an Image file from protoc

Instead of using buf build, you can directly use protoc to build a FileDescriptorSet (which is also an Image), although you have to be careful to set the correct flags.

# assuming your only proto_path is "."
$ protoc -I . --include_imports -o image.bin $(find . -name '*.proto')

Note that if you do not include imports (which will only be the Well-Known Types at this point), you should specify --exclude-imports when using buf check breaking - Buf cannot derive what files are imports from protoc-produced FileDescriptorSets.

buf check breaking --against image.bin --exclude-imports

Error Output

If there are errors, they will be printed out in a file:line:column:message format by default, where references are to your current Protobuf files.

Note that buf check breaking is able to produce references to your current files even if a type is moved between files, so for example if we moved the Date message to another file, Buf would reference the location within this file instead. It will also attempt to use an enclosing type for deleted references - for example, if a field is deleted, Buf will reference the enclosing message if still present. and if a nested message is deleted, Buf will reference the enclosing message as well.

For example, from the tour:

$ buf check breaking --against "https://github.com/googleapis/googleapis/archive/${GOOGLEAPIS_COMMIT}.tar.gz#strip_components=1"
google/type/date.proto:50:3:Field "3" on message "Date" changed type from "int32" to "string".

Output can also be printed as JSON. From the tour:

$ buf check breaking --against "https://github.com/googleapis/googleapis/archive/${GOOGLEAPIS_COMMIT}.tar.gz#strip_components=1" --error-format=json
{"path":"google/type/date.proto","start_line":50,"start_column":3,"end_line":50,"end_column":9,"type":"FIELD_SAME_TYPE","message":"Field \"3\" on message \"Date\" changed type from \"int32\" to \"string\"."}

Suppress File Discovery

By default, Buf builds all files under configuration per the build config. You can instead manually specify the file paths to build and check. This is an advanced feature intended to be used Bazel integration - it is better to let Buf discover all files under management and handle this for you in general, especially with breaking change detection using the FILE category as file existence matters.

Breaking change detection will be limited to the given files if specified.

File paths should be the actual file paths when your input is a Source format, and the root-relative file paths when your input is an Image format (rare).

# our input is a Source format, ie "." which is a directory, the default
# our --against is an Image format, but this is fine - Buf will figure out what
# files to compare based on the root-relative paths
$ buf check breaking --against image.bin --path path/to/foo.proto --path path/to/bar.proto

Combine this with an inline configuration override to make the only file I/O performed by Buf being the reading of your specified .proto files.

$ buf check breaking --against image.bin --path path/to/foo.proto --path path/to/bar.proto --config '{"breaking":{"use":["WIRE_JSON"]}}'

File comparisons

This is really only relevant when using the FILE category - all other breaking checkers do not care about file paths.

Buf will compare files based on their root-relative paths, that is Buf will strip the root directory from each file, and use the resulting path as the comparison point. For example, if on your master branch you had a single root ".", and you now move all your .proto files to proto/, buf check breaking will still happily compare the two versions of your schema - on master, all paths will be relative to "." for the purposes of comparison, and in your current state, all paths will be relative to "proto".

Ignore vendored Protobuf files

A lot of builds will have vendored Protobuf files that you want to may want to exclude from breaking change detection. We do not recommend excluding them - your API will still break regardless of whether a definition is considered to be your own or vendored. However, if you want to do this, use the ignore configuration option. For example, you may build against googleapis for grpc-gateway. The following represents how to handle this situation:

version: v1beta1
build:
roots:
# the proto/ directory contains your Protobuf definitions
# the googleapis/ directory contains the vendored Protobuf definitions from grpc-gateway
# note that per the documentation for roots, roots cannot overlap, so if you have this
# situation you will need a proto/ directory
- proto
- vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis
lint:
ignore:
# per the breaking configuration docs, the paths for breaking ignores have the root
# directory stripped so that this configuration works for both sources and images
# google/api and google/rpc are the directories within googleapis/ that we want to ignore
- google/api
- google/rpc

Docker

Buf ships a Docker image bufbuild/buf that allows you to use Buf as part of your Docker workflow. For example:

$ docker run \
--volume "$(pwd):/workspace" \
--workdir /workspace \
bufbuild/buf check breaking --against '.git#branch=master'

Convoluted Input Examples

Due to the nature of Inputs, Buf will happily compare just about anything. Most of these examples are useless or overly complicated, but you may have an advanced use case and we want to provide these to demonstrate the capabilities of Buf.

Compare a remote git repository to a remote archive with a configuration override

We've tested this example locally, you should be able to copy/paste this into your terminal.

$ buf check breaking \
"https://github.com/googleapis/googleapis.git" \
--against "https://github.com/googleapis/googleapis/archive/b89f7fa5e7cc64e9e38a59c97654616ad7b5932d.tar.gz#strip_components=1" \
--config '{"breaking":{"use":["PACKAGE"]}}'
google/cloud/asset/v1/assets.proto:27:1:File option "cc_enable_arenas" changed from "false" to "true".

To explicitly target the master branch:

$ buf check breaking \
"https://github.com/googleapis/googleapis.git#branch=master" \
--against "https://github.com/googleapis/googleapis/archive/b89f7fa5e7cc64e9e38a59c97654616ad7b5932d.tar.gz#strip_components=1" \
--config '{"breaking":{"use":["PACKAGE"]}}'
google/cloud/asset/v1/assets.proto:27:1:File option "cc_enable_arenas" changed from "false" to "true".

Read an image from stdin and compare the current state to the current state

As like any other buf command, buf check breaking can read input from stdin. This is useful if, for example, you are downloading an image file from a private location. As a fun but useless example, let's build an Image of our current state and output to stdout, then compare against the input from stdin. This should always pass as it is comparing the current state to the current state:

$ buf build -o - | buf check breaking --against -