Top Related Projects
Go support for Google's protocol buffers
[Deprecated] Protocol Buffers for Go with Gadgets
A simple RPC framework with protobuf service definitions
GoMock is a mocking framework for the Go programming language.
The Go language implementation of gRPC. HTTP/2 based RPC
Protocol Buffer Validation - Being replaced by github.com/bufbuild/protovalidate
Quick Overview
The golang/protobuf repository is the official Protocol Buffers implementation for the Go programming language. It provides tools and libraries for working with Protocol Buffers, a language-agnostic data serialization format developed by Google. This project enables Go developers to efficiently serialize and deserialize structured data for storage or transmission.
Pros
- High performance and efficient serialization
- Language-agnostic, allowing interoperability with other programming languages
- Strong typing and automatic code generation
- Backward and forward compatibility for evolving data structures
Cons
- Steeper learning curve compared to simpler formats like JSON
- Requires additional tooling and build steps
- Less human-readable than text-based formats
- Limited support for dynamic structures compared to some alternatives
Code Examples
- Defining a Protocol Buffer message:
syntax = "proto3";
package example;
message Person {
string name = 1;
int32 age = 2;
repeated string hobbies = 3;
}
- Generating Go code from the .proto file:
protoc --go_out=. person.proto
- Using the generated Go code:
package main
import (
"fmt"
"log"
"google.golang.org/protobuf/proto"
pb "path/to/generated/package"
)
func main() {
person := &pb.Person{
Name: "Alice",
Age: 30,
Hobbies: []string{"reading", "hiking"},
}
data, err := proto.Marshal(person)
if err != nil {
log.Fatal("Marshaling error: ", err)
}
newPerson := &pb.Person{}
if err := proto.Unmarshal(data, newPerson); err != nil {
log.Fatal("Unmarshaling error: ", err)
}
fmt.Println("Unmarshaled person:", newPerson)
}
Getting Started
-
Install the Protocol Compiler (protoc):
# For macOS using Homebrew brew install protobuf # For Ubuntu sudo apt-get install protobuf-compiler
-
Install the Go protocol buffers plugin:
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
-
Add the following to your Go module:
go get google.golang.org/protobuf
-
Define your .proto file, generate Go code, and start using Protocol Buffers in your Go project as shown in the code examples above.
Competitor Comparisons
Go support for Google's protocol buffers
Pros of protobuf-go
- Officially maintained by Google, ensuring better long-term support and compatibility
- Improved API design and more idiomatic Go code
- Better performance in certain scenarios, especially for large messages
Cons of protobuf-go
- Breaking changes from protobuf, requiring code updates when migrating
- Some features and third-party extensions may not be fully compatible
- Steeper learning curve for developers familiar with the older API
Code Comparison
protobuf:
import "github.com/golang/protobuf/proto"
data, err := proto.Marshal(message)
err = proto.Unmarshal(data, message)
protobuf-go:
import "google.golang.org/protobuf/proto"
data, err := proto.Marshal(message)
err = proto.Unmarshal(data, message)
While the basic usage remains similar, protobuf-go introduces new packages and types for more advanced operations, such as reflection and custom options handling. The new API provides better type safety and more idiomatic Go code, but may require significant changes when migrating existing projects.
Overall, protobuf-go is recommended for new projects and those willing to update their codebase, while protobuf remains a viable option for maintaining existing implementations or when compatibility with certain third-party libraries is required.
[Deprecated] Protocol Buffers for Go with Gadgets
Pros of gogo/protobuf
- Faster serialization and deserialization performance
- Additional customization options and extensions
- More flexible code generation with custom types and methods
Cons of gogo/protobuf
- Less official support and maintenance compared to golang/protobuf
- Potential compatibility issues with future protobuf updates
- Steeper learning curve due to additional features and complexity
Code Comparison
protobuf:
type Person struct {
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
Age int32 `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
}
gogo/protobuf:
type Person struct {
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
Age int32 `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
// gogo/protobuf extensions
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
The main difference in the code is the additional fields added by gogo/protobuf for optimization and caching purposes. These extensions contribute to the improved performance but also increase the complexity of the generated code.
A simple RPC framework with protobuf service definitions
Pros of Twirp
- Simpler and more lightweight RPC framework
- Built-in support for HTTP/JSON clients
- Easier to set up and use, with less boilerplate code
Cons of Twirp
- Less mature and widely adopted compared to protobuf
- Limited to HTTP 1.1 transport
- Fewer language implementations available
Code Comparison
Twirp service definition:
service Haberdasher {
rpc MakeHat(Size) returns (Hat);
}
Protobuf service definition:
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
Twirp client usage:
client := haberdasher.NewHaberdasherProtobufClient("http://localhost:8080", &http.Client{})
hat, err := client.MakeHat(ctx, &haberdasher.Size{Inches: 12})
Protobuf client usage:
conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
defer conn.Close()
c := pb.NewGreeterClient(conn)
r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: "world"})
Both Twirp and protobuf offer efficient serialization and RPC capabilities, but Twirp focuses on simplicity and HTTP compatibility, while protobuf provides a more comprehensive and flexible solution with broader language support and transport options.
GoMock is a mocking framework for the Go programming language.
Pros of mock
- Specifically designed for mocking in Go, making it easier to create and use mocks in unit tests
- Provides a more intuitive API for creating and manipulating mocks compared to protobuf
- Integrates well with Go's testing package and supports various matching and expectation features
Cons of mock
- Limited to mocking functionality, whereas protobuf offers a broader range of serialization and RPC capabilities
- May require more setup and configuration for complex mocking scenarios compared to protobuf's simpler approach
- Less suitable for cross-language or cross-platform development compared to protobuf's language-agnostic nature
Code Comparison
mock:
ctrl := gomock.NewController(t)
defer ctrl.Finish()
mockObj := NewMockInterface(ctrl)
mockObj.EXPECT().SomeMethod(gomock.Any()).Return(42)
protobuf:
message MyMessage {
string name = 1;
int32 id = 2;
}
func (m *MyMessage) ProtoMessage() {}
The code snippets demonstrate the different focus areas of the two libraries. mock is centered around creating and manipulating mock objects for testing, while protobuf is designed for defining message structures and serialization.
The Go language implementation of gRPC. HTTP/2 based RPC
Pros of grpc-go
- Full gRPC implementation with advanced features like streaming, load balancing, and interceptors
- Integrated with the broader gRPC ecosystem, including tools and plugins
- Better performance optimizations for large-scale distributed systems
Cons of grpc-go
- Steeper learning curve due to more complex API and concepts
- Heavier dependency footprint compared to the lightweight protobuf package
- May be overkill for simple projects that only need basic protobuf serialization
Code Comparison
protobuf:
import "github.com/golang/protobuf/proto"
data, err := proto.Marshal(message)
err = proto.Unmarshal(data, message)
grpc-go:
import "google.golang.org/grpc"
conn, err := grpc.Dial(address, grpc.WithInsecure())
client := pb.NewServiceClient(conn)
response, err := client.Method(ctx, request)
Summary
protobuf is a lightweight library focused on Protocol Buffers serialization, while grpc-go is a comprehensive gRPC implementation. protobuf is simpler and more suitable for basic serialization needs, whereas grpc-go offers a complete RPC framework with advanced features for building distributed systems. The choice between them depends on project requirements, complexity, and scalability needs.
Protocol Buffer Validation - Being replaced by github.com/bufbuild/protovalidate
Pros of protoc-gen-validate
- Provides more extensive validation options for Protocol Buffer messages
- Supports cross-language validation code generation
- Offers a more flexible and customizable validation framework
Cons of protoc-gen-validate
- Requires additional setup and configuration compared to protobuf
- May have a steeper learning curve for developers new to custom validation rules
- Potentially increased code complexity due to additional validation layers
Code Comparison
protobuf:
message Person {
string name = 1;
int32 age = 2;
}
protoc-gen-validate:
message Person {
string name = 1 [(validate.rules).string = {min_len: 1, max_len: 100}];
int32 age = 2 [(validate.rules).int32 = {gte: 0, lte: 120}];
}
In this example, protoc-gen-validate allows for more granular validation rules directly in the Protocol Buffer definition, whereas protobuf relies on separate validation logic implemented in the application code.
protoc-gen-validate excels in scenarios requiring complex validation rules and cross-language support, while protobuf remains a simpler, more straightforward option for basic Protocol Buffer implementations. The choice between the two depends on the specific needs of the project, with protoc-gen-validate offering more robust validation capabilities at the cost of increased complexity.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Go support for Protocol Buffers
This module
(github.com/golang/protobuf
)
contains Go bindings for protocol buffers.
It has been superseded by the
google.golang.org/protobuf
module, which contains an updated and simplified API,
support for protobuf reflection, and many other improvements.
We recommend that new code use the google.golang.org/protobuf
module.
Versions v1.4 and later of github.com/golang/protobuf
are implemented
in terms of google.golang.org/protobuf
.
Programs which use both modules must use at least version v1.4 of this one.
See the developer guide for protocol buffers in Go for a general guide for how to get started using protobufs in Go.
See release note documentation for more information about individual releases of this project.
See documentation for the next major revision for more information about the purpose, usage, and history of this project.
Package index
Summary of the packages provided by this module:
proto
: Packageproto
provides functions operating on protobuf messages such as cloning, merging, and checking equality, as well as binary serialization and text serialization.jsonpb
: Packagejsonpb
serializes protobuf messages as JSON.ptypes
: Packageptypes
provides helper functionality for protobuf well-known types.ptypes/any
: Packageany
is the generated package forgoogle/protobuf/any.proto
.ptypes/empty
: Packageempty
is the generated package forgoogle/protobuf/empty.proto
.ptypes/timestamp
: Packagetimestamp
is the generated package forgoogle/protobuf/timestamp.proto
.ptypes/duration
: Packageduration
is the generated package forgoogle/protobuf/duration.proto
.ptypes/wrappers
: Packagewrappers
is the generated package forgoogle/protobuf/wrappers.proto
.ptypes/struct
: Packagestructpb
is the generated package forgoogle/protobuf/struct.proto
.protoc-gen-go/descriptor
: Packagedescriptor
is the generated package forgoogle/protobuf/descriptor.proto
.protoc-gen-go/plugin
: Packageplugin
is the generated package forgoogle/protobuf/compiler/plugin.proto
.protoc-gen-go
: Theprotoc-gen-go
binary is a protoc plugin to generate a Go protocol buffer package.
Reporting issues
The issue tracker for this project is located here.
Please report any issues with a sufficient description of the bug or feature request. Bug reports should ideally be accompanied by a minimal reproduction of the issue. Irreproducible bugs are difficult to diagnose and fix (and likely to be closed after some period of time). Bug reports must specify the version of the Go protocol buffer module and also the version of the protocol buffer toolchain being used.
Contributing
This project is open-source and accepts contributions. See the contribution guide for more information.
Compatibility
This module and the generated code are expected to be stable over time. However, we reserve the right to make breaking changes without notice for the following reasons:
- Security: A security issue in the specification or implementation may come to light whose resolution requires breaking compatibility. We reserve the right to address such issues.
- Unspecified behavior: There are some aspects of the protocol buffer specification that are undefined. Programs that depend on unspecified behavior may break in future releases.
- Specification changes: It may become necessary to address an inconsistency, incompleteness, or change in the protocol buffer specification, which may affect the behavior of existing programs. We reserve the right to address such changes.
- Bugs: If a package has a bug that violates correctness, a program depending on the buggy behavior may break if the bug is fixed. We reserve the right to fix such bugs.
- Generated additions: We reserve the right to add new declarations to
generated Go packages of
.proto
files. This includes declared constants, variables, functions, types, fields in structs, and methods on types. This may break attempts at injecting additional code on top of what is generated byprotoc-gen-go
. Such practice is not supported by this project. - Internal changes: We reserve the right to add, modify, and remove
internal code, which includes all unexported declarations, the
generator
package, and all packages underinternal
.
Any breaking changes outside of these will be announced 6 months in advance to protobuf@googlegroups.com.
Top Related Projects
Go support for Google's protocol buffers
[Deprecated] Protocol Buffers for Go with Gadgets
A simple RPC framework with protobuf service definitions
GoMock is a mocking framework for the Go programming language.
The Go language implementation of gRPC. HTTP/2 based RPC
Protocol Buffer Validation - Being replaced by github.com/bufbuild/protovalidate
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot