Build configuration overview
Buf is configured through a
buf.yaml file that should be checked in to the
root of your repository. Buf will automatically read this file if present. Configuration
can also be provided via the command-line flag
--config, which accepts a path
.yaml file, or direct JSON or YAML data.
All Buf operations that use your local
.proto files as Input rely on a valid
build configuration. This configuration tells Buf where to search for
.proto files, and
how to handle imports. As opposed to
protoc, where all
.proto files are manually
specified on the command-line,
buf operates by recursively discovering all
files under configuration and building them.
The following is an example of all configuration options for
build section only has two options:
This is a list of the directories that contain your
.proto files. The directory paths
must be relative to the root of your repository, and cannot point to a location outside
of your repository. They also represent the root of your import paths within your
For those familiar with
roots corresponds to your
--proto_paths, aliased as
protoc - that is, these are the directories that the compiler uses to search for imports.
As an example, if your repository has two files,
proto/foo/baz/baz.proto, we recommend having the following configuration:
baz.proto wants to import
bar.proto, it does so relative to
Generally, for a given Protobuf repository, we expect one main root that contains all your
.proto files. For most repositories, we recommend this be a directory
proto/. This is
the pattern that the buf-example repository uses.
Other repositories that have no vendored
.proto files may choose to use the root of the
repository as the only root. This is what googleapis
does, as an example - all imports derive from the root of the repository.
This is a list of the directories to ignore from file discovery. Any directories added to this list will be completely skipped and considered not part of the Protobuf schema. We do not recommend using this option in general, however in some situations it is unavoidable.
There are two additional requirements that
buf imposes on your
.proto file structure
for compilation to succeed that are not enforced by
protoc, both of which are very
important for successful modern Protobuf development across a number of languages
1. Roots must not overlap, that is one root can not be a sub-directory of another root.
For example, the following is not a valid configuration:
This is important to make sure that across all your
.proto files, imports are consistent
In the above example, for a given file
foo/bar/bar.proto, it would be valid to import
this file as either
bar.proto. Having inconsistent imports leads
to a number of major issues across the Protobuf plugin ecosystem.
.proto file paths must be unique relative to the roots.
For example, consider the following configuration:
Given the above configuration, it is invalid to have the following two files:
This results in two files having the path
baz/baz.proto. Given the following third file
Which file is being imported? Is it
bar/baz/baz.proto? The answer depends
on the order of the
-I flags given to
protoc, or (if Buf didn't error in this scenario
pre-compilation, which Buf does) the order of the imports given to the internal compiler. If
the authors are being honest, we can't remember if it's the first
-I or second
-I that wins -
we have outlawed this in our own builds for a long time.
While the above example is relatively contrived, the common error that comes up is when you
.proto files. For example, grpc-gateway has it's own copy of the google.api definitions it needs. While these are usually in sync, the
can change. If we allowed the following:
Which copy of
google/api/*.proto wins? The answer is no one wins, so this is not allowed.
Manual proto_path and file specification
In most scenarios, it is best to let Buf run file discovery on it's own - this will result
in the least error-prone usage of Buf, and the most consistent results from the linter
and breaking change detector. However, there are scenarios where you do not want Buf
to search for
.proto files, or you want to specify the entire build configuration
on the command line, for example with Bazel integration.
Buf allows this through the
--path flag and the various configuration flags on each
--path flag can be specified multiple times, and specifies the exact
files or directories to use for Protobuf compilation. For every Buf command, configuration
can also be passed on the command line.
As an example, let's say we want to replicate
protoc's build behavior with
(described in more detail in the next section). Assume we have:
- A single root,
- Two files,
Can be replicated by the following