Top Related Projects
A modern formatting library
Protocol Buffers - Google's data interchange format
MessagePack implementation for C and C++ / msgpack.org[C/C++]
Cap'n Proto serialization/RPC system - core tools and C++ library
Apache Thrift
JSON for Modern C++
Quick Overview
FlatBuffers is an efficient cross-platform serialization library developed by Google. It allows you to access serialized data without parsing/unpacking it first, while still supporting data structure evolution (forward/backward compatibility).
Pros
- High performance: Zero-copy access to serialized data without parsing
- Memory efficient: Minimal memory allocation during serialization and deserialization
- Cross-platform: Supports multiple programming languages and platforms
- Schema evolution: Allows for forward and backward compatibility
Cons
- Learning curve: Requires understanding of the FlatBuffers schema language
- Limited built-in types: May require custom serialization for complex data types
- Larger file size: Serialized data can be larger compared to some alternatives
- Less human-readable: Binary format is not easily readable without tools
Code Examples
Creating a FlatBuffer:
flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("John Doe");
auto person = CreatePerson(builder, name, 30);
builder.Finish(person);
Reading from a FlatBuffer:
auto person = GetPerson(buffer_pointer);
std::string name = person->name()->str();
int32_t age = person->age();
Mutating a FlatBuffer:
auto person = GetMutablePerson(mutable_buffer_pointer);
person->mutate_age(31);
Getting Started
-
Install FlatBuffers:
git clone https://github.com/google/flatbuffers.git cd flatbuffers cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release make
-
Define your schema (e.g.,
person.fbs
):table Person { name:string; age:int; } root_type Person;
-
Compile the schema:
./flatc --cpp person.fbs
-
Use the generated code in your C++ project:
#include "person_generated.h" // Use FlatBuffers as shown in the code examples above
Competitor Comparisons
A modern formatting library
Pros of fmt
- Focused on string formatting and output, providing a more intuitive and type-safe alternative to printf
- Header-only library, making it easy to integrate into projects
- Supports a wide range of formatting options and custom types
Cons of fmt
- Limited to string formatting and output, not a general-purpose serialization solution
- May have slightly higher compile times due to heavy template usage
- Requires C++11 or later, which might be an issue for older codebases
Code Comparison
fmt:
#include <fmt/core.h>
std::string result = fmt::format("Hello, {}!", "world");
fmt::print("The answer is {}.", 42);
FlatBuffers:
#include "monster_generated.h"
flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("Orc");
auto monster = CreateMonster(builder, name, 0, 300);
builder.Finish(monster);
Summary
fmt is a powerful string formatting library, while FlatBuffers is a serialization system. They serve different purposes, with fmt excelling in text output and FlatBuffers focusing on efficient data serialization. Choose fmt for string manipulation and formatting, and FlatBuffers for cross-platform data exchange and storage.
Protocol Buffers - Google's data interchange format
Pros of Protocol Buffers
- More mature and widely adopted, with extensive language support
- Built-in support for backwards compatibility and schema evolution
- Smaller message sizes for complex data structures with many optional fields
Cons of Protocol Buffers
- Slower serialization and deserialization compared to FlatBuffers
- Requires parsing the entire message before accessing any data
- Larger binary size due to additional runtime library
Code Comparison
Protocol Buffers:
message Person {
required string name = 1;
optional int32 age = 2;
repeated string hobbies = 3;
}
FlatBuffers:
table Person {
name:string;
age:int32;
hobbies:[string];
}
Both examples define a simple Person structure with similar fields. Protocol Buffers uses keywords like required
, optional
, and repeated
, while FlatBuffers uses a more straightforward table definition.
FlatBuffers offers zero-copy deserialization, allowing direct access to data without parsing the entire message. This results in faster performance for large datasets or when only partial data access is needed. However, Protocol Buffers provides better support for schema evolution and backwards compatibility, making it more suitable for systems with frequently changing data structures or long-term data storage requirements.
MessagePack implementation for C and C++ / msgpack.org[C/C++]
Pros of msgpack-c
- Simpler and more lightweight serialization format
- Wider language support and ecosystem
- Faster serialization and deserialization for small data structures
Cons of msgpack-c
- Less efficient for larger, more complex data structures
- Lacks built-in schema evolution and backwards compatibility features
- No direct support for zero-copy reads
Code Comparison
msgpack-c:
msgpack_pack_array(&pk, 3);
msgpack_pack_int(&pk, 1);
msgpack_pack_true(&pk);
msgpack_pack_str(&pk, 5);
msgpack_pack_str_body(&pk, "Hello", 5);
FlatBuffers:
auto name = builder.CreateString("Hello");
auto monster = CreateMonster(builder, 1, true, name);
builder.Finish(monster);
FlatBuffers requires a schema definition and generates code, while msgpack-c uses a more dynamic approach. FlatBuffers offers stronger typing and better performance for larger datasets, but msgpack-c is simpler for small, dynamic data structures.
Both libraries are widely used and have their strengths. FlatBuffers excels in scenarios requiring high performance and schema evolution, while msgpack-c is more suitable for simpler, lightweight serialization needs across multiple languages.
Cap'n Proto serialization/RPC system - core tools and C++ library
Pros of Cap'n Proto
- Zero-copy deserialization, resulting in faster parsing
- Built-in RPC system for distributed computing
- More expressive schema language with advanced features like generics
Cons of Cap'n Proto
- Larger message size due to padding and alignment requirements
- Steeper learning curve and more complex API
- Less widespread adoption compared to FlatBuffers
Code Comparison
Cap'n Proto schema:
struct Person {
name @0 :Text;
age @1 :UInt32;
address @2 :Address;
}
FlatBuffers schema:
table Person {
name:string;
age:uint32;
address:Address;
}
Both Cap'n Proto and FlatBuffers offer efficient serialization formats for structured data. Cap'n Proto excels in scenarios requiring zero-copy deserialization and distributed computing, while FlatBuffers provides a simpler API and smaller message sizes. The choice between them depends on specific project requirements, such as performance needs, message size constraints, and desired features like RPC support.
Apache Thrift
Pros of Thrift
- Supports more programming languages than FlatBuffers
- Provides built-in RPC (Remote Procedure Call) functionality
- Offers a more mature ecosystem with broader adoption
Cons of Thrift
- Generally slower serialization and deserialization compared to FlatBuffers
- Larger message sizes, especially for small data structures
- Requires full parsing of data before accessing any fields
Code Comparison
Thrift IDL:
struct Person {
1: string name
2: i32 age
3: string email
}
FlatBuffers IDL:
table Person {
name:string;
age:int32;
email:string;
}
Both IDLs define a simple Person structure with similar fields. The main difference is in syntax and how the fields are numbered (Thrift) or not (FlatBuffers).
FlatBuffers allows for zero-copy deserialization and direct access to fields without parsing the entire message, which can lead to performance benefits in certain scenarios. Thrift, on the other hand, offers more language support and built-in RPC capabilities, making it potentially more suitable for complex distributed systems.
JSON for Modern C++
Pros of json
- Easier to read and write JSON data directly
- Wider language support and ecosystem
- More flexible for dynamic data structures
Cons of json
- Slower parsing and serialization
- Larger data size, especially for numeric values
- Less type safety compared to strongly-typed schemas
Code Comparison
json:
#include <nlohmann/json.hpp>
using json = nlohmann::json;
json j = {
{"name", "John"},
{"age", 30},
{"city", "New York"}
};
std::string serialized = j.dump();
FlatBuffers:
#include "person_generated.h"
flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("John");
auto person = CreatePerson(builder, name, 30, "New York");
builder.Finish(person);
uint8_t *buf = builder.GetBufferPointer();
FlatBuffers requires a schema definition and code generation, while json allows for more dynamic data manipulation. FlatBuffers offers better performance and smaller data size, but at the cost of flexibility and ease of use. The choice between the two depends on specific project requirements, such as performance needs, data structure complexity, and development workflow preferences.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
FlatBuffers
FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility.
Quick Start
-
Build the compiler for flatbuffers (
flatc
)Use
cmake
to create the build files for your platform and then perform the compilation (Linux example).cmake -G "Unix Makefiles" make -j
-
Define your flatbuffer schema (
.fbs
)Write the schema to define the data you want to serialize. See monster.fbs for an example.
-
Generate code for your language(s)
Use the
flatc
compiler to take your schema and generate language-specific code:./flatc --cpp --rust monster.fbs
Which generates
monster_generated.h
andmonster_generated.rs
files. -
Serialize data
Use the generated code, as well as the
FlatBufferBuilder
to construct your serialized buffer. (C++
example) -
Transmit/store/save Buffer
Use your serialized buffer however you want. Send it to someone, save it for later, etc...
-
Read the data
Use the generated accessors to read the data from the serialized buffer.
It doesn't need to be the same language/schema version, FlatBuffers ensures the data is readable across languages and schema versions. See the
Rust
example reading the data written byC++
.
Documentation
Go to our landing page to browse our documentation.
Supported operating systems
- Windows
- macOS
- Linux
- Android
- And any others with a recent C++ compiler (C++ 11 and newer)
Supported programming languages
Code generation and runtime libraries for many popular languages.
- C
- C++ - snapcraft.io
- C# - nuget.org
- Dart - pub.dev
- Go - go.dev
- Java - Maven
- JavaScript - NPM
- Kotlin
- Lobster
- Lua
- PHP
- Python - PyPI
- Rust - crates.io
- Swift - swiftpackageindex
- TypeScript - NPM
- Nim
Versioning
FlatBuffers does not follow traditional SemVer versioning (see rationale) but rather uses a format of the date of the release.
Contribution
- FlatBuffers Issues Tracker to submit an issue.
- stackoverflow.com with
flatbuffers
tag for any questions regarding FlatBuffers.
To contribute to this project, see CONTRIBUTING.
Community
Security
Please see our Security Policy for reporting vulnerabilities.
Licensing
Flatbuffers is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Top Related Projects
A modern formatting library
Protocol Buffers - Google's data interchange format
MessagePack implementation for C and C++ / msgpack.org[C/C++]
Cap'n Proto serialization/RPC system - core tools and C++ library
Apache Thrift
JSON for Modern C++
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot