Skip to main content

Setup

If you are using GitHub Actions, you can skip this guide and refer to the GitHub Actions guide instead.

Continuous Integration/Continuous Deployment (CI/CD) is a software development practice that automates building, testing, and deploying software. If you are working with Protobuf, then buf should be part of all three of these development stages.

This guide illustrates how to integrate buf into general CI/CD solutions, such as CircleCI and TravisCI. If you are using GitHub Actions, you can skip this guide and refer to the GitHub Actions guide instead.

This guide is also supplemented by the buf-example repository, which provides a functional example for integrating buf into CircleCI, TravisCI, or GitHub Actions. For a quick solution that leverages a Makefile, please refer to buf-example!

Installation#

This is demonstrated in buf-example, so please refer to that repository for a functional example.

The first step is to get buf running on your CI/CD worker. In order to do so, you'll need an install script. buf can be downloaded from a release or built from source.

install.sh
#!/bin/bash
PROJECT=<your-project-name># Use your desired buf versionBUF_VERSION=1.0.0-rc1# buf will be cached to ~/.cache/your-project-name.CACHE_BIN=$HOME/.cache/$(PROJECT)
curl -sSL \    "https://github.com/bufbuild/buf/releases/download/v$BUF_VERSION/buf-$(shell uname -s)-$(shell uname -m)" \    -o "$CACHE_BIN/buf"chmod +x "$CACHE_BIN/buf"

This script sends a request to the buf Github Releases using curl for the given BUF_VERSION and operating system. The binary is then given executable permission.

Running lint and breaking change detection#

This is demonstrated in buf-example, so please refer to that repository for a functional example.

To run lint checks with your job, simply add buf lint to it and you're good to go!

If your buf.yaml is defined at the root of your repository, the command is as simple as:

buf lint

If, on the other hand, your buf.yaml is defined in a nested directory, such as the proto directory, the command looks like the following:

buf lint proto

For buf breaking, the process is similar, but be sure to set the full https or ssh remote as the target. If your buf.yaml is defined at the root of your repository, the command looks like the following:

buf breaking --against "https://github.com/<your-org>/<your-repo>.git#branch=main"# orbuf breaking --against "ssh://git@github.com/<your-org>/<your-repo>.git#branch=main"

Again, if your buf.yaml is defined in a nested directory, such as the proto directory, the command looks like the following (notice the subdir parameter):

buf breaking proto --against "https://github.com/<your-org>/<your-repo>.git#branch=main,subdir=proto"# orbuf breaking proto --against "ssh://git@github.com/<your-org>/<your-repo>.git#branch=main,subdir=proto"

If you are on TravisCI or CircleCI they do not clone any branches outside of the one being tests, so this allows buf to clone using the remote and run the breaking change detector.

CI authentication (Optional)#

If you wish to authenticate a CI/CD job to access the BSR (for example, push a module, create tags, etc.), we recommend you store your BUF_TOKEN in your CI/CD provider's secret environment variable storage.

For example:

You can then access the token in your job using an environment variable, which allows you to create a .netrc file for your job during setup. Here's an example assuming you've stored your token as BUF_API_TOKEN:

echo -e "machine buf.build\npassword ${BUF_API_TOKEN}" >> ~/.netrc

For more details on authenticating to the BSR, please see Authentication.

Wrapping up#

Now that you've set up buf to run lint checks and detect breaking changes in your CI/CD environment, your APIs will always remain consistent, and you won't need to waste any more time understanding the complex backwards compatibility rules to ensure that you never break your customers.