Top Related Projects
Quick Overview
gogo/protobuf is an extension of the Go Protocol Buffers (protobuf) implementation, offering enhanced functionality and performance optimizations. It provides additional features and code generation options beyond the standard protobuf library, allowing for more efficient serialization and deserialization of structured data.
Pros
- Improved performance compared to standard protobuf implementation
- Additional features and customization options for generated code
- Backward compatibility with standard protobuf
- Active community and ongoing development
Cons
- Increased complexity compared to standard protobuf
- Potential compatibility issues with other protobuf implementations
- Steeper learning curve for advanced features
- May require additional setup and configuration
Code Examples
- Basic message definition and usage:
syntax = "proto3";
package example;
option go_package = "example";
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
message Person {
string name = 1 [(gogoproto.moretags) = "json:\"name,omitempty\""];
int32 age = 2;
}
- Using custom types:
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
message CustomMessage {
bytes uuid = 1 [(gogoproto.customtype) = "github.com/gogo/protobuf/test.UUID"];
int64 timestamp = 2 [(gogoproto.customtype) = "github.com/gogo/protobuf/types.Timestamp"];
}
- Generating faster marshaling/unmarshaling code:
syntax = "proto3";
package example;
option go_package = "example";
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
option (gogoproto.marshaler_all) = true;
option (gogoproto.unmarshaler_all) = true;
option (gogoproto.sizer_all) = true;
message FastMessage {
string data = 1;
repeated int32 numbers = 2;
}
Getting Started
-
Install gogo/protobuf:
go get github.com/gogo/protobuf/protoc-gen-gofast
-
Define your .proto file with gogo/protobuf options.
-
Generate Go code:
protoc --gofast_out=. your_file.proto
-
Use the generated code in your Go project:
import "path/to/generated/package" // Use the generated structs and methods msg := &example.YourMessage{ Field: "value", } data, err := msg.Marshal()
Competitor Comparisons
Go support for Google's protocol buffers
Pros of golang/protobuf
- Official Google-maintained implementation
- Better compatibility with standard protobuf features
- More frequent updates and maintenance
Cons of golang/protobuf
- Generally slower performance
- Lacks some advanced features and optimizations
- More verbose code generation
Code Comparison
golang/protobuf:
type Person struct {
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
Age int32 `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
}
gogo/protobuf:
type Person struct {
Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
Age int32 `protobuf:"varint,2,opt,name=age,proto3" json:"age,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
Summary
golang/protobuf is the official implementation, offering better compatibility and maintenance. However, gogo/protobuf provides performance optimizations and additional features. The code generation is similar, but gogo/protobuf includes extra fields for optimization. Choose based on your project's needs for performance vs. standard compatibility.
A simple RPC framework with protobuf service definitions
Pros of Twirp
- Simpler and more lightweight than gRPC, making it easier to learn and implement
- Built-in support for both JSON and Protobuf encoding, offering flexibility
- Designed for HTTP/1.1, making it compatible with a wider range of clients and servers
Cons of Twirp
- Less feature-rich compared to gRPC, which may limit its use in complex scenarios
- Smaller community and ecosystem compared to Protobuf, potentially leading to fewer resources and tools
Code Comparison
Twirp:
service Haberdasher {
rpc MakeHat(Size) returns (Hat);
}
Protobuf:
service Haberdasher {
rpc MakeHat(Size) returns (Hat);
}
Key Differences
- Twirp focuses on simplicity and ease of use, while Protobuf (with gRPC) offers more advanced features
- Twirp is designed specifically for HTTP/1.1, whereas Protobuf can be used with various transport protocols
- Twirp provides a more opinionated structure, while Protobuf offers more flexibility in implementation
Use Cases
- Twirp: Ideal for smaller projects or microservices where simplicity and quick implementation are priorities
- Protobuf: Better suited for large-scale, complex systems that require advanced features and cross-language support
Community and Ecosystem
- Protobuf has a larger, more established community with extensive tooling and support
- Twirp has a smaller but growing community, focused on simplicity and ease of use
GoMock is a mocking framework for the Go programming language.
Pros of mock
- Specifically designed for mocking in Go, making it easier to create and use mocks in unit tests
- Integrates well with Go's testing package and provides a simple API for creating mock objects
- Supports method stubbing and expectation setting, allowing for more flexible test scenarios
Cons of mock
- Limited to mocking functionality, whereas protobuf offers a broader range of features for data serialization and RPC
- May require more manual setup for complex mocking scenarios compared to protobuf's code generation capabilities
- Less suitable for cross-language development and communication between different systems
Code Comparison
mock:
ctrl := gomock.NewController(t)
defer ctrl.Finish()
mockObj := NewMockInterface(ctrl)
mockObj.EXPECT().SomeMethod(gomock.Any()).Return(42)
protobuf:
message Person {
string name = 1;
int32 age = 2;
}
service PersonService {
rpc GetPerson(PersonRequest) returns (Person) {}
}
Summary
While mock is specialized for creating mock objects in Go unit tests, protobuf is a more comprehensive solution for data serialization and RPC communication. mock excels in test-driven development scenarios, offering easy integration with Go's testing framework. protobuf, on the other hand, provides a language-agnostic approach to defining data structures and services, making it more suitable for cross-platform development and communication between different systems.
The Go language implementation of gRPC. HTTP/2 based RPC
Pros of grpc-go
- Official gRPC implementation for Go, ensuring compatibility and support
- Comprehensive features for full gRPC functionality
- Regular updates and maintenance from the gRPC team
Cons of grpc-go
- Generally slower performance compared to gogo/protobuf
- More complex API and usage
- Larger memory footprint
Code Comparison
grpc-go:
import "google.golang.org/grpc"
conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
if err != nil {
log.Fatalf("Failed to connect: %v", err)
}
defer conn.Close()
gogo/protobuf:
import "github.com/gogo/protobuf/proto"
data, err := proto.Marshal(&myMessage)
if err != nil {
log.Fatal("marshaling error: ", err)
}
The grpc-go example shows how to establish a gRPC connection, while the gogo/protobuf example demonstrates message serialization. grpc-go provides a more comprehensive gRPC implementation, while gogo/protobuf focuses on efficient protobuf encoding/decoding.
gogo/protobuf offers better performance and smaller generated code size, making it suitable for high-performance scenarios. However, grpc-go provides the full gRPC feature set and is maintained by the official gRPC team, ensuring long-term support and compatibility with the gRPC ecosystem.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
[Deprecated] Protocol Buffers for Go with Gadgets
gogoprotobuf is a fork of golang/protobuf with extra code generation features.
This code generation is used to achieve:
- fast marshalling and unmarshalling
- more canonical Go structures
- goprotobuf compatibility
- less typing by optionally generating extra helper code
- peace of mind by optionally generating test and benchmark code
- other serialization formats
Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this issue
Release v1.3.0
The project has updated to release v1.3.0. Check out the release notes here.
With this new release comes a new internal library version. This means any newly generated *pb.go files generated with the v1.3.0 library will not be compatible with the old library version (v1.2.1). However, current *pb.go files (generated with v1.2.1) should still work with the new library.
Please make sure you manage your dependencies correctly when upgrading your project. If you are still using v1.2.1 and you update your dependencies, one of which could include a new *pb.go (generated with v1.3.0), you could get a compile time error.
Our upstream repo, golang/protobuf, also had to go through this process in order to update their library version. Here is a link explaining hermetic builds.
Users
These projects use gogoprotobuf:
- etcd - blog - sample proto file
- spacemonkey - blog
- badoo - sample proto file
- mesos-go - sample proto file
- heka - the switch from golang/protobuf to gogo/protobuf when it was still on code.google.com
- cockroachdb - sample proto file
- go-ipfs - sample proto file
- rkive-go - sample proto file
- dropbox
- srclib - sample proto file
- adyoulike
- cloudfoundry - sample proto file
- kubernetes - go2idl built on top of gogoprotobuf
- dgraph - release notes - benchmarks
- centrifugo - release notes - blog
- docker swarmkit - sample proto file
- nats.io - go-nats-streaming
- tidb - Communication between tidb and tikv
- protoactor-go - vanity command that also generates actors from service definitions
- containerd - vanity command with custom field names that conforms to the golang convention.
- nakama
- proteus
- carbonzipper stack
- sendgrid
- zero-os/0-stor
- go-spacemesh
- cortex - sample proto file
- Apache SkyWalking APM - Istio telemetry receiver based on Mixer bypass protocol
- Hyperledger Burrow - a permissioned DLT framework
- IOV Weave - a blockchain framework - sample proto files
Please let us know if you are using gogoprotobuf by posting on our GoogleGroup.
Mentioned
- Cloudflare - go serialization talk - Albert Strasheim
- GopherCon 2014 Writing High Performance Databases in Go by Ben Johnson
- alecthomas' go serialization benchmarks
- Go faster with gogoproto - Agniva De Sarker
- Evolution of protobuf (Gource Visualization) - Landon Wilkins
- Creating GopherJS Apps with gRPC-Web - Johan Brandhorst
- So you want to use GoGo Protobuf - Johan Brandhorst
- Advanced gRPC Error Usage - Johan Brandhorst
- gRPC Golang Course on Udemy - Stephane Maarek
Preparing for GopherCon UK 2022
Getting Started
There are several ways to use gogoprotobuf, but for all you need to install go and protoc. After that you can choose:
- Speed
- More Speed and more generated code
- Most Speed and most customization
Installation
To install it, you must first have Go (at least version 1.6.3 or 1.9 if you are using gRPC) installed (see http://golang.org/doc/install). Latest patch versions of 1.12 and 1.15 are continuously tested.
Next, install the standard protocol buffer implementation from https://github.com/google/protobuf. Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.14.0 are continuously tested.
Speed
Install the protoc-gen-gofast binary
go get github.com/gogo/protobuf/protoc-gen-gofast
Use it to generate faster marshaling and unmarshaling go code for your protocol buffers.
protoc --gofast_out=. myproto.proto
This does not allow you to use any of the other gogoprotobuf extensions.
More Speed and more generated code
Fields without pointers cause less time in the garbage collector. More code generation results in more convenient methods.
Other binaries are also included:
protoc-gen-gogofast (same as gofast, but imports gogoprotobuf)
protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields)
protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods)
Installing any of these binaries is easy. Simply run:
go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/{binary}
go get github.com/gogo/protobuf/gogoproto
These binaries allow you to use gogoprotobuf extensions. You can also use your own binary.
To generate the code, you also need to set the include path properly.
protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto
To use proto files from "google/protobuf" you need to add additional args to protoc.
protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\
Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \
myproto.proto
Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen".
Most Speed and most customization
Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize. gogoprotobuf also offers more serialization formats and generation of tests and even more methods.
Please visit the extensions page for more documentation.
Install protoc-gen-gogo:
go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/jsonpb
go get github.com/gogo/protobuf/protoc-gen-gogo
go get github.com/gogo/protobuf/gogoproto
GRPC
It works the same as golang/protobuf, simply specify the plugin. Here is an example using gofast:
protoc --gofast_out=plugins=grpc:. my.proto
See https://github.com/gogo/grpc-example for an example of using gRPC with gogoprotobuf and the wider grpc-ecosystem.
License
This software is licensed under the 3-Clause BSD License ("BSD License 2.0", "Revised BSD License", "New BSD License", or "Modified BSD License").
Top Related Projects
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot