protoreflect

Reflection (Rich Descriptors) for Go Protocol Buffers

1,363

175

1,363

View on GitHub

Top Related Projects

protobuf

5,664

[Deprecated] Protocol Buffers for Go with Gadgets

protobuf-go

2,905

Go support for Google's protocol buffers

protoc-gen-validate

3,818

Protocol Buffer Validation - Being replaced by github.com/bufbuild/protovalidate

Quick Overview

The jhump/protoreflect project is a Go library that provides a set of tools for working with Protocol Buffers (protobuf), a popular data serialization format. It offers a comprehensive set of APIs for parsing, manipulating, and generating protobuf-based code, making it a valuable resource for developers working with protobuf in Go.

Pros

Comprehensive Protobuf Support: The library provides a wide range of functionality for working with protobuf, including parsing, reflection, and code generation.
Flexibility: The library is designed to be highly flexible, allowing developers to customize and extend its behavior to fit their specific needs.
Performance: The library is optimized for performance, making it a suitable choice for high-performance applications.
Active Development: The project is actively maintained and regularly updated, ensuring that it stays up-to-date with the latest protobuf specifications and best practices.

Cons

Steep Learning Curve: The library's extensive functionality and flexibility can make it challenging for newcomers to get started, especially if they are not familiar with protobuf or Go.
Limited Documentation: While the project has a decent amount of documentation, some areas may be lacking in detailed explanations and examples.
Dependency on Protobuf: The library is tightly coupled with the protobuf ecosystem, which means that developers who are not using protobuf may not find it useful.
Potential Performance Overhead: Depending on the specific use case, the library's flexibility and comprehensive feature set may introduce some performance overhead compared to more lightweight protobuf libraries.

Code Examples

Here are a few code examples demonstrating the usage of the jhump/protoreflect library:

Parsing a Protobuf Message:

import (
    "fmt"
    "github.com/jhump/protoreflect/desc"
    "github.com/jhump/protoreflect/dynamic"
)

// Parse a protobuf message from a byte slice
msg, err := dynamic.AsDynamicMessage([]byte{...})
if err != nil {
    // handle error
}

// Access the message fields
fmt.Println(msg.GetFieldByName("name").GetStringValue())

Generating Protobuf Code:

import (
    "github.com/jhump/protoreflect/desc/protoparse"
    "github.com/jhump/protoreflect/desc/protoprint"
)

// Parse a protobuf file
fileDesc, err := protoparse.Parser{}.ParseFile("example.proto")
if err != nil {
    // handle error
}

// Generate Go code from the file descriptor
goCode, err := protoprint.PrintProtoAsGoCode(fileDesc)
if err != nil {
    // handle error
}

// Print the generated Go code
fmt.Println(goCode)

Manipulating Protobuf Messages:

import (
    "github.com/jhump/protoreflect/dynamic"
)

// Create a new protobuf message
msg := dynamic.NewMessage(&desc.MessageDescriptor{})

// Set a field value
msg.SetFieldByName("name", "John Doe")

// Serialize the message to bytes
bytes, err := msg.Marshal()
if err != nil {
    // handle error
}

Reflecting on Protobuf Descriptors:

import (
    "fmt"
    "github.com/jhump/protoreflect/desc"
)

// Load a protobuf file descriptor
fileDesc, err := desc.LoadFileDescriptor("example.proto")
if err != nil {
    // handle error
}

// Iterate over the message descriptors
for _, msgDesc := range fileDesc.GetMessageTypes() {
    fmt.Println(msgDesc.GetName())
}

Getting Started

To get started with the jhump/protoreflect library, follow these steps:

Install the library using Go's package manager:

go get github.com/jhump/protoreflect

Import the necessary packages in your Go code:

import (

Competitor Comparisons

protobuf

5,664

[Deprecated] Protocol Buffers for Go with Gadgets

Pros of gogo/protobuf

Offers significant performance improvements and optimizations over standard protobuf
Provides additional code generation options and customization features
Supports custom types and extensions for enhanced flexibility

Cons of gogo/protobuf

Less actively maintained compared to protoreflect
May have compatibility issues with newer protobuf versions
Steeper learning curve due to additional features and complexity

Code Comparison

protoreflect:

message := &myproto.MyMessage{}
err := proto.Unmarshal(data, message)
if err != nil {
    return err
}

gogo/protobuf:

message := &myproto.MyMessage{}
err := message.Unmarshal(data)
if err != nil {
    return err
}

Summary

protoreflect focuses on reflection capabilities and compatibility with standard protobuf, while gogo/protobuf emphasizes performance optimizations and additional features. protoreflect is more actively maintained and aligns closely with the official protobuf implementation. gogo/protobuf offers performance benefits but may require more effort to integrate and maintain in projects.

Choose protoreflect for better long-term support and standard compatibility, or gogo/protobuf for performance-critical applications willing to trade some compatibility for speed.

protobuf-go

2,905

Go support for Google's protocol buffers

Pros of protobuf-go

Official implementation by Google, ensuring compatibility and regular updates
Optimized performance for Go-specific use cases
Seamless integration with other Google Cloud services and tools

Cons of protobuf-go

Limited flexibility for advanced use cases and custom modifications
Stricter adherence to Protocol Buffers specification, potentially limiting some Go-specific optimizations

Code Comparison

protobuf-go:

import "google.golang.org/protobuf/proto"

func main() {
    message := &MyMessage{...}
    data, err := proto.Marshal(message)
}

protoreflect:

import "github.com/jhump/protoreflect/dynamic"

func main() {
    message := dynamic.NewMessage(messageDescriptor)
    data, err := message.Marshal()
}

protobuf-go provides a more straightforward API for basic Protocol Buffers operations, while protoreflect offers greater flexibility and reflection capabilities. protobuf-go is ideal for standard use cases and integration with Google services, whereas protoreflect is better suited for advanced scenarios requiring dynamic message handling or custom Protocol Buffers implementations.

protoc-gen-validate

3,818

Protocol Buffer Validation - Being replaced by github.com/bufbuild/protovalidate

Pros of protoc-gen-validate

Focused specifically on generating validation code for Protocol Buffers
Supports multiple programming languages (Go, Java, Python, etc.)
Provides a rich set of validation rules and constraints

Cons of protoc-gen-validate

Limited to validation functionality only
Requires additional setup and integration with the protoc compiler
May have a steeper learning curve for complex validation scenarios

Code Comparison

protoc-gen-validate:

message User {
  string email = 1 [(validate.rules).string.email = true];
  int32 age = 2 [(validate.rules).int32 = {gte: 18, lte: 99}];
}

protoreflect:

import "github.com/jhump/protoreflect/desc"

func validateUser(user *User) error {
    md, _ := desc.LoadMessageDescriptorForMessage(user)
    // Custom validation logic using reflection
}

Summary

protoc-gen-validate is a specialized tool for generating validation code for Protocol Buffers, supporting multiple languages and offering a rich set of validation rules. It's ideal for projects requiring strict data validation but may have a steeper learning curve.

protoreflect, on the other hand, provides a more general-purpose reflection API for Protocol Buffers in Go. It offers greater flexibility for custom operations but requires more manual implementation for validation tasks.

Choose protoc-gen-validate for streamlined, multi-language validation generation, or protoreflect for broader Protocol Buffer manipulation capabilities in Go.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Protocol Buffer and gRPC Reflection

This repo builds on top of the reflection capabilities in the Protobuf runtime for Go and also provides reflection APIs for gRPC as well.

[!NOTE] Version 2.0.0 is still a work in progress. It is basically feature complete, but still needs more tests for the new functionality.

You can try it out by getting a pre-release version:
go get github.com/jhump/protoreflect/v2@v2.0.0-beta.2
Note that the APIs may change a little bit between now and a final v2.0.0 release. Also note that some packages in v2 still need more tests, so you may find some bugs. But that should mostly be for new functionality. If you're just trying to update your code from v1 of this repo, those packages should be rock-solid and least likely to see any further API changes.

Descriptors and Reflection Utilities

The protoreflect package in the Protobuf Go runtime provides the Descriptor interface and implementations of it that correspond to each of the descriptor types. These types are effectively smart wrappers around the generated Protobuf types in the descriptorpb package. These wrappers make descriptors much more useful and easier to use.

This repo provides some additional packages for using and interacting with descriptors.

import "github.com/jhump/protoreflect/v2/protoprint"

The protoprint package allows for printing of descriptors to .proto source files. This is effectively the inverse of a parser/compiler (such as the protocompile package.) Combined with the protobuilder package, this is a useful tool for programmatically generating protocol buffer sources.

Read more â«

import "github.com/jhump/protoreflect/v2/protobuilder"

The protobuilder package allows for programmatic construction of rich descriptors. Descriptors can be constructed programmatically by creating trees of descriptor protos and using the protodesc package to link those into rich descriptors. But constructing a valid tree of descriptor protos is far from trivial.

So this package provides generous API to greatly simplify that task. It also allows for converting rich descriptors into builders, which means you can programmatically modify/tweak existing descriptors.

Read more â«

import "github.com/jhump/protoreflect/v2/protoresolve"

The protoresolve package provides named interfaces for many kinds of resolvers. It also provides a Resolver interface that acts like a union of the various resolver interfaces and unifies both descriptor resolvers and type resolvers. The former returns descriptor instances; the latter returns types (often implemented by the dynamicpb package). These interfaces provide a comprehensive set of types for resolving elements in Protobuf schemas and effectively extend the APIs in the protoregistry package provided by the Protobuf Go runtime.

Read more â«

import "github.com/jhump/protoreflect/v2/protomessage"

The protomessage package contains helpers for work with proto.Message instances from generic and/or dynamic code.

Read more â«

import "github.com/jhump/protoreflect/v2/protodescs"

The protodescs package contains miscellaneous helpers for working with descriptors.

Read more â«

Source Code Info

Generated Protobuf types in Go do not include "source code information". Source code information is data that comes from the original Protobuf source file that defined messages and includes things like position information (i.e. the filename, line, and column on which a message, enum, or service was defined) and comments.

This repo includes some APIs to help work with source code info and also a mechanism (and Proto plugin) for restoring the source code information to the descriptors embedded in generated Go code.

import "github.com/jhump/protoreflect/v2/sourceinfo"

The sourceinfo package contains APIs that for retrieving descriptors for generated types that include source code info. When generating Go code, source code information is not preserved. But if you also generate code using the included protoc-gen-gosrcinfo plugin and query for the descriptors using this package, you can access that information. The most immediate use of this information is to provide comments for services, methods, and types to dynamic RPC clients that use the gRPC server reflection service.

Read more â«

import "github.com/jhump/protoreflect/v2/sourceloc"

The sourceloc package contains helpers for working with instances of protoreflect.SourceLocation and protoreflect.SourcePath.

Read more â«

Dynamic RPC Stubs

The dynamicpb package in the Protobuf Go runtime provides a dynamic message implementation. It implements proto.Message but is backed by a message descriptor and a map of fields->values, instead of a generated struct. This is useful for acting generically with protocol buffer messages, without having to generate and link in Go code for every kind of message. This is particularly useful for general-purpose tools that need to operate on arbitrary Protobuf schemas. This is made possible by having the tools load descriptors at runtime.

This repo provides capabilities on top of dynamicpb to not only use message schemas dynamically but to also use RPC schemas dynamically. This enables invoking RPCs without having any generated code for the RPC service to be used.

import "github.com/jhump/protoreflect/v2/grpcdynamic"

The grpcdynamic package provides the dynamic stub implementation. The stub can be used to issue RPC methods using method descriptors instead of generated client interfaces.

Read more â«

gRPC Server Reflection

import "github.com/jhump/protoreflect/v2/grpcreflect"

The grpcreflect package provides an easy-to-use client for the gRPC reflection service, making it much easier to query for and work with the schemas of remote services.

It also provides some helper methods for querying for rich service descriptors for the services registered in a gRPC server.

Read more â«

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot