Top Related Projects
Go support for Google's protocol buffers
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Your Swiss Army Knife for Protocol Buffers
Documentation generator plugin for Google Protocol Buffers
Quick Overview
Buf is a modern Protocol Buffers ecosystem, providing tools for API design, linting, breaking change detection, and code generation. It aims to simplify and streamline the development process for Protocol Buffer-based APIs, offering a more user-friendly and efficient alternative to traditional protoc-based workflows.
Pros
- Simplified workflow for Protocol Buffer development and management
- Powerful linting and breaking change detection capabilities
- Supports remote generation and caching of generated code
- Integrates well with existing Protocol Buffer ecosystems
Cons
- Learning curve for developers familiar with traditional protoc workflows
- May require changes to existing build processes and CI/CD pipelines
- Limited support for some less common Protocol Buffer features
- Relatively new project, still evolving and stabilizing
Code Examples
- Initializing a Buf project:
buf mod init
This command initializes a new Buf module in the current directory.
- Linting Protocol Buffer files:
buf lint
This command runs the Buf linter on your Protocol Buffer files, checking for style and best practice violations.
- Detecting breaking changes:
buf breaking --against '.git#branch=main'
This command checks for breaking changes between the current branch and the main branch.
Getting Started
To get started with Buf, follow these steps:
-
Install Buf:
brew install bufbuild/buf/buf
-
Initialize a Buf project:
buf mod init
-
Create a
buf.yaml
configuration file:version: v1 lint: use: - DEFAULT breaking: use: - FILE
-
Run Buf commands:
buf lint buf generate buf breaking --against '.git#branch=main'
These steps will set up a basic Buf project and demonstrate some of its core functionalities.
Competitor Comparisons
Go support for Google's protocol buffers
Pros of protobuf
- Mature and widely adopted in the Go ecosystem
- Directly supported and maintained by the Go team
- Extensive documentation and community resources
Cons of protobuf
- Limited built-in linting and formatting capabilities
- Lacks advanced features for managing multiple proto files and dependencies
- Requires additional tools for a complete protobuf workflow
Code Comparison
protobuf:
import "google.golang.org/protobuf/proto"
message := &MyProtoMessage{Field: "value"}
data, err := proto.Marshal(message)
buf:
import "github.com/bufbuild/buf/cmd/buf/command"
cmd := command.NewRoot()
err := cmd.Execute()
Key Differences
- buf focuses on providing a comprehensive toolkit for Protocol Buffers, including linting, formatting, and dependency management
- protobuf is primarily a library for working with Protocol Buffers in Go
- buf offers a more streamlined workflow for managing proto files across projects
- protobuf provides lower-level access to Protocol Buffer internals
Use Cases
- Choose protobuf for direct integration with Go projects and when lower-level control is needed
- Opt for buf when working on larger projects with multiple proto files, or when seeking a more robust development workflow
Community and Ecosystem
- protobuf has a larger user base and more third-party integrations
- buf is gaining popularity for its modern approach to Protocol Buffer development
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Pros of gRPC
- Mature and widely adopted framework with extensive language support
- Comprehensive ecosystem with tools for load balancing, authentication, and monitoring
- High-performance, low-latency communication using HTTP/2
Cons of gRPC
- Steeper learning curve and more complex setup compared to Buf
- Requires more boilerplate code for basic implementations
- Limited browser support without additional proxies or libraries
Code Comparison
gRPC (Python):
import grpc
from example_pb2 import ExampleRequest
from example_pb2_grpc import ExampleServiceStub
channel = grpc.insecure_channel('localhost:50051')
stub = ExampleServiceStub(channel)
response = stub.ExampleMethod(ExampleRequest(name='John'))
Buf (Go):
import (
"context"
"github.com/bufbuild/connect-go"
"github.com/bufbuild/connect-grpcreflect-go"
)
client := connect.NewClient[ExampleServiceClient](http.DefaultClient, "http://localhost:8080")
resp, err := client.ExampleMethod(context.Background(), connect.NewRequest(&ExampleRequest{Name: "John"}))
Summary
gRPC offers a robust, high-performance framework with extensive language support and a comprehensive ecosystem. However, it comes with a steeper learning curve and more complex setup. Buf provides a simpler, more streamlined approach to building and managing Protocol Buffers-based APIs, making it easier to get started but potentially sacrificing some of the advanced features and ecosystem support that gRPC offers.
Your Swiss Army Knife for Protocol Buffers
Pros of Prototool
- More mature project with longer history and established user base
- Offers linting and formatting capabilities out of the box
- Provides a single binary for multiple protobuf-related tasks
Cons of Prototool
- Less active development and maintenance compared to Buf
- Limited support for modern protobuf features and ecosystem integrations
- Slower performance for large protobuf projects
Code Comparison
Prototool configuration:
lint:
rules:
add:
- ENUM_NAMES_UPPER_SNAKE_CASE
remove:
- ENUM_NAMES_CAMEL_CASE
Buf configuration:
version: v1
lint:
use:
- DEFAULT
enum_zero_value_suffix: _UNSPECIFIED
breaking:
use:
- FILE
Prototool focuses on a more traditional approach to protobuf management, while Buf offers a more modern and comprehensive solution. Buf provides better performance, more extensive ecosystem support, and a stronger focus on breaking change detection. However, Prototool may be preferred by users who need a simpler, all-in-one tool for basic protobuf tasks and are comfortable with its established workflow.
Documentation generator plugin for Google Protocol Buffers
Pros of protoc-gen-doc
- Focused specifically on generating documentation for Protocol Buffers
- Supports multiple output formats (HTML, Markdown, JSON)
- Lightweight and easy to integrate into existing protoc workflows
Cons of protoc-gen-doc
- Limited to documentation generation, lacking broader protobuf tooling features
- Requires manual integration with protoc, potentially more setup work
- Less active development and community support compared to Buf
Code Comparison
protoc-gen-doc usage:
protoc --doc_out=./docs --doc_opt=html,index.html protos/*.proto
Buf usage:
buf generate
Key Differences
Buf is a comprehensive Protocol Buffers ecosystem tool, offering features like linting, breaking change detection, and a remote schema registry. protoc-gen-doc, on the other hand, is a specialized tool for generating documentation from protobuf files.
Buf aims to simplify the entire protobuf workflow, including code generation and management, while protoc-gen-doc focuses solely on documentation generation.
Buf has a more active development community and frequent updates, whereas protoc-gen-doc has a more stable but less frequently updated codebase.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Buf
The buf
CLI is the best tool for working with Protocol Buffers. It provides:
- A linter that enforces good API design choices and structure.
- A breaking change detector that enforces compatibility at the source code or wire level.
- A generator that invokes your plugins based on configurable templates.
- A formatter that formats your Protobuf files in accordance with industry standards.
- Integration with the Buf Schema Registry, including full dependency management.
Installation
Homebrew
You can install buf
using Homebrew (macOS or Linux):
brew install bufbuild/buf/buf
This installs:
- The
buf
,protoc-gen-buf-breaking
, andprotoc-gen-buf-lint
binaries - Shell completion scripts for Bash, Fish, Powershell, and zsh
Other methods
For other installation methods, see our official documentation, which covers:
- Installing
buf
via npm - Installing
buf
on Windows - Using
buf
as a Docker image - Installing as a binary, from a tarball, and from source through GitHub Releases
- Verifying releases using a minisign public key
Usage
Buf's help interface provides summaries for commands and flags:
buf --help
For more comprehensive usage information, consult Buf's documentation, especially these guides:
buf breaking
buf build
buf generate
buf lint
buf format
buf registry
(for using the BSR)
CLI breaking change policy
We will never make breaking changes within a given major version of the CLI. Once buf
reaches v1.0, you can expect no breaking changes until v2.0. But as we have no plans to ever release a v2.0, we will likely never break the buf
CLI.
This breaking change policy does not apply to commands behind the
buf beta
gate, and you should expect breaking changes to commands likebuf beta registry
. The policy does go into effect, however, when those commands or flags are elevated out of beta.
Our goals for Protobuf
Buf's goal is to replace the current paradigm of API development, centered around REST/JSON, with a schema-driven paradigm. Defining APIs using an IDL provides numerous benefits over REST/JSON, and Protobuf is by far the most stable and widely adopted IDL in the industry. We've chosen to build on this widely trusted foundation rather than creating a new IDL from scratch.
But despite its technical merits, actually using Protobuf has long been more challenging than it needs to be. The Buf CLI and the BSR are the cornerstones of our effort to change that for good and to make Protobuf reliable and easy to use for service owners and clients alikeâin other words, to create a modern Protobuf ecosystem.
While we intend to incrementally improve on the buf
CLI and the BSR, we're confident that the basic groundwork for such an ecosystem is already in place.
The Buf Schema Registry
The Buf Schema Registry (BSR) is a SaaS platform for managing your Protobuf APIs. It provides a centralized registry and a single source of truth for all of your Protobuf assets, including not just your .proto
files but also remote plugins. Although the BSR provides an intuitive browser UI, buf
enables you to perform most BSR-related tasks from the command line, such as pushing Protobuf sources to the registry and managing users and repositories.
The BSR is not required to use
buf
. We've made the core features of thebuf
CLI available to all Protobuf users.
More advanced CLI features
While buf
's core features should cover most use cases, we've included some more advanced features to cover edge cases:
- Automatic file discovery. Buf walks your file tree and builds your
.proto
files in accordance with your supplied build configuration, which means that you no longer need to manually specify--proto_paths
. You can still, however, specify.proto
files manually through CLI flags in cases where file discovery needs to be disabled. - Fine-grained rule configuration for linting and breaking changes. While we do have recommended defaults, you can always select the exact set of rules that your use case requires, with 40 lint rules and 53 breaking change rules available.
- Configurable error formats for CLI output.
buf
outputs information infile:line:column:message
form by default for each lint error and breaking change it encounters, but you can also select JSON and, in the near future, JUnit output. - Editor integration driven by
buf
's granular error output. We currently provide linting integrations for both Vim and Visual Studio Code but we plan to support other editors, such as Emacs and JetBrains IDEs like IntelliJ and GoLand, in the future. - Universal Input targeting. Buf enables you to perform actions like linting and breaking change detection not just against local
.proto
files but also against a broad range of other Inputs, such as tarballs and ZIP files, remote Git repositories, and pre-built image files. - Speed. Buf's internal Protobuf compiler compiles your Protobuf sources using all available cores without compromising deterministic output, which is considerably faster than
protoc
. This allows for near-instantaneous feedback, which is of special importance for features like editor integration.
Next steps
Once you've installed buf
, we recommend completing the CLI tutorial, which provides a broad but hands-on overview of the core functionality of the CLI. The tour takes about 10 minutes to complete.
After completing the tour, check out the remainder of the docs for your specific areas of interest.
Community
For help and discussion around Protobuf, best practices, and more, join us on Slack.
For updates on the Buf CLI, follow this repo on GitHub.
For feature requests, bugs, or technical questions, email us at dev@buf.build. For general inquiries or inclusion in our upcoming feature betas, email us at info@buf.build.
Top Related Projects
Go support for Google's protocol buffers
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Your Swiss Army Knife for Protocol Buffers
Documentation generator plugin for Google Protocol Buffers
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot