As discussed in the overview , buf categorizes breaking rules into four main categories:
FILE: Generated source code breaking changes on a per-file basis, that is changes that would break the generated stubs where definitions cannot be moved across files. This makes sure that for languages such as C++ and Python where header files are included, your source code will never break for a given Protobuf change. This category also verifies wire and JSON compatibility.
PACKAGE: Generated source code breaking changes on a per-package basis, that is changes that would break the generated stubs, but only accounting for package-level changes. This is useful for languages such as Java (with
option java_multiple_files = true;set) or Golang where it is fine to move Protobuf types across files, as long as they stay within the same Protobuf package. This category also verifies wire and JSON compatibility.
WIRE: Wire breaking changes, that is changes that would break wire compatibility, including checks to make sure you reserve deleted types of which re-use in the future could cause wire incompatibilities.
WIRE_JSON: Wire breaking changes and JSON breaking changes, that is changes that would break either wire compatibility or JSON compatibility. This mostly extends
WIREto include field and enum value names.
Choosing a category
As opposed to lint rules, you generally will not mix and exclude specific breaking change rules, instead choosing one of the categories:
Choose a category based on the following:
- If you distribute your generated source code outside of a monorepo in any capacity, or want
to make sure that consumers of the generated source code will not have broken builds, use
FILEif you use (or might use) any languages that generate header files (such as C++ or Python), or
PACKAGEif you only use languages that generate code on a per-package basis (such as Golang). If in doubt, choose
- If you only use Protobuf within a monorepo and always re-generate, and are OK with refactoring
code that consumes the generated source code, use
WIREif you are sure that you will never use the JSON representation of your Protobuf messages. Use
WIRE_JSONif there is any JSON usage. Generally, we'd recommend using
WIRE_JSONif you go this route (this basically just has the effect of not allowing re-use of field and enum value names).
We'd recommend using
FILE is the default. Generally, the case
where you really never use a given Protobuf schema anywhere but a single monorepo for
an entire organization (and all the organization's external customers) is rarer than
Also as opposed to lint rules, there is not a strict subset relationship between the main categories in terms of what rules belong to what categories. However, in terms of strictness, the order is:
FILE is the strictest, and
WIRE is the most permissive. Even though
there is no strict subset relationship, you can safely assume that passing the
implies you would pass the
PACKAGE rules, and that passing the
PACKAGE rules implies
you would pass the
WIRE_JSON rules, and using the
WIRE_JSON rules implies you
would pass the
WIRE rules. There is no need to specify all of them.
As an example of how this works, consider the rules
ENUM_NO_DELETE is in the
FILE category, and checks that for each file,
no enum is deleted.
PACKAGE_NO_DELETE is in the
PACKAGE category, and checks that for a given
package, no enum is deleted, however enums are allowed to move between files within a package.
Given these definitions, and given that a file does not change it's package (which is checked
FILE_SAME_PACKAGE, also in the
FILE category), it is obvious that passing
As another example, consider
FIELD_NO_DELETE, a rule in the
categories that checks that no field is deleted from a given message. Consider this as
FIELD_NO_DELETE_UNLESS_NUMBER_RESERVED (part of the
WIRE_JSON categories) and
FIELD_NO_DELETE_UNLESS_NAME_RESERVED (part of the
WIRE_JSON category), rules that
do not allow field deletion unless the number or name is reserved. Clearly, if
passes (that is, no field is deleted), both
FIELD_NO_DELETE_UNLESS_NAME_RESERVED would pass as well.
What we left out (we think nothing, except custom options)
We think the below rules represent a complete view of what is and isn't breaking with respect to Protobuf schemas - we cover every available field within a FileDescriptorSet as of protobuf v3.11.4, and will cover additional fields as added. If we missed something, please let us know, urgently.
We did leave out custom options however, almost by definition - there's no way for us to know the effects of your custom options. We may, however, add the google.api options in the future if there is sufficient demand, especially google.api.http.
We'll discuss each of the rules here.
These check that no enums, messages or services are deleted from a given file. Deleting an enum, message or service will delete the corresponding generated type, which could be referenced in source code. Instead of deleting these types, deprecate them:
These have the same effect as their non-prefixed counterparts above, except that this
verifies that these types are not deleted from a given package, while letting types
move between files. For example, if
foo2.proto both have
enum Bar could move from
foo2.proto without representing
a breaking change.
This checks that no file is deleted. Deleting a file will result in it's generated header file being deleted as well, which could break source code.
This checks that a given file has the same
package value. Changing the package
value will result in a ton of issues downstream in various languages, and for
FILE category, this will effectively result in any types declared within
that file being considered deleted.
This checks that no packages are deleted. This basically checks that at least one file in your previous schema has a package declared for every package declared in your current schema. Deleting packages means that all types within those packages are deleted, and even though each of these types are checked, this is more of a sanity check.
These check that no enum value or message field is deleted. Deleting an enum value or message field will result in the corresponding value or field being deleted from the generated source code, which could be referenced. Instead of deleting these, deprecate them:
These check that no enum value or message field is deleted without reserving the number. While deleting an enum value or message field is not directly a wire-breaking change, re-using these numbers in the future will result in either bugs (in the case of enums) or actual wire incompatibilities (in the case of messages, if the type differs). This is a JSON breaking change for enum values if enum values are serialized as ints (which is an option). Protobuf provides the ability to reserve numbers to prevent them being re-used in the future. For example:
Note that deprecating a field instead of deleting it has the same effect as reserving the field (as well as reserving the name for JSON), so this is what we recommend.
These check that no enum value or message field is deleted without reserving the name. This is the JSON-equivalent of reserving the number - JSON uses field names instead of numbers (this is optional for enum fields, but allowed). Generally you will want to reserve both the number and the name. For example:
Note is is significantly easier to just deprecated enum values and message fields.
This checks that no RPC is deleted from a service. Doing so is not a wire-breaking change (although client calls fail if a server does not implement a given RPC), however existing source code may reference a given RPC. Instead of deleting an RPC, deprecate it.
This checks that no oneof is deleted from a message. Various languages generate types for oneofs, which will no longer be present if deleted.
This checks that a file does not switch between
going to/from unset (which assumes
proto2) to set to
proto3. Changing the syntax
results in differences in generated code for many languages.
These check that each of these file options do not change values between versions of your Protobuf schema. Changing any of these values will result in differences in your generated source code.
Note that you may not use any or all of these languages in your own development, and that's more than fine - if you don't set any of these options, none of these rules will ever break. You may not have been aware some of these options existed - if so, put them in your rear view mirror.
This checks that a given enum value has the same name for each enum value number. For example
You cannot change
FOO_ONE = 1 to
FOO_TWO = 1. Doing so will result in potential JSON
incompatibilites and broken source code.
Note that for enums with
allow_alias set, this verifies that the set of names in the
current definition covers the set of names in the previous definition. For example,
the new definition
// new is compatible with
// old is not compatible with
This checks that a given field has the same value for the ctype option. This affects the C++ generator. This is a Google-internal field option, so generally you won't have this set, and this rule will have no effect.
This checks that a field has the same type. Changing the type of a field can affect the type in the generated source code, wire compatibility, and JSON compatibility. Note that technically, it is possible to interchange some scalar types, however most of these result in generated source code changes anyways, and affect JSON compatibility - instead of worrying about these, just don't change your field types.
Note that with maps, Buf currently has an issue where you may get a weird set of error messages when changing a field to/from a map and some other type, denoting that the type of the field changed from "FieldNameEntry" to something else. This is due to how maps are implemented in Protobuf, where every map is actually just a repeated field of an implicit message of name "FieldNameEntry". Correcting these error messages isn't impossible, and it's on our roadmap, but it just hasn't been high priority - Buf will still properly detect this change and output an error, so the pass/fail decision remains the same.
This checks that no field changes it's label, i.e.
optional, required, repeated. Changing to/from
optional/required and repeated will be a generated source code and JSON breaking change. Changing
to/from optional and repeated is actually not a wire-breaking change, however changing to/from
optional and required is. Given that it's unlikely to be advisable in any situation to change your
label, and that there is only one exception, we find it best to just outlaw this entirely.
This checks that no field moves into or out of a oneof, or changes the oneof it is a part of. Doing so is almost always a generated source code breaking change. Technically there are exceptions with regards to wire compatibility, but the rules are not something you should concern yourself with, and it is safer to just never change a field's presence inside or outside a given oneof.
This checks that the field name for a given field number does not change. For example, you
int64 foo = 1; to
int64 bar = 1;. This affects generated source code,
but also affects JSON compatibility as JSON uses field names for serialization. This does
not affect wire compatibility, however we generally don't recommend changing field names.
This checks that the
json_name field option does not change, which would break
JSON compatibility. While not a generated source code breaking change in general, it is
conceivable that some Protobuf plugins may generate code based on this option, and having
this as part of the
PACKAGE groups also fulfills that the
are supersets of the
These check that no reserved number range or reserved name is deleted from any enum or message. Deleting a reserved value that future versions of your Protobuf schema can then use names or numbers in these ranges, and if these ranges are reserved, it was because an enum value or field was deleted.
Note that moving from i.e.
reserved 3 to 6; to
reserved 2 to 8; would technically be fine,
however Buf will still fail in this case - making sure all ranges are covered is truly a pain,
we have no other excuse. We could fix this in the future. For now, just do
reserved 3 to 6, 2, 7 to
8; to pass breaking change detection.
This checks that no extension range is deleted from any message. While this won't have any effect on your generated source code, deleting an extension range can result in compile errors for downstream Protobuf schemas, and is generally not recommended. Note that extensions are a proto2-only construct, so this has no effect for proto3.
This checks that the message_set_wire_format message option is the same. Since this is a proto1 construct, we congratulate you if you are using this for any current Protobuf schema, as you are a champion of maintaining backwards compatible APIs over many years. Instead of failing breaking change detection, perhaps you should get an award.
This checks that the no_standard_descriptor_accessor message option is not changed from
false/unset to true. Changing this option to true will result in the
descriptor() accessor is not
generated in certain languages, which is a generated source code breaking change. Protobuf has
issues with fields that are named "descriptor", of any capitalization and with any number of
underscores before and after "descriptor". Don't name fields this. Before v1.0, we may add a lint
rule that verifies this.
These check that RPC signatures do not change. Doing so would break both generated source code and over-the-wire RPC calls.
This checks that the idempotency_level RPC option does not change. Doing so can result in different HTTP verbs being used.