Convert Figma logo to code with AI

google logoflatbuffers

FlatBuffers: Memory Efficient Serialization Library

23,057
3,223
23,057
160

Top Related Projects

20,408

A modern formatting library

65,113

Protocol Buffers - Google's data interchange format

MessagePack implementation for C and C++ / msgpack.org[C/C++]

11,517

Cap'n Proto serialization/RPC system - core tools and C++ library

10,310

Apache Thrift

42,154

JSON for Modern C++

Quick Overview

FlatBuffers is an efficient cross-platform serialization library developed by Google. It allows you to access serialized data without parsing/unpacking it first, while still supporting data structure evolution (forward/backward compatibility).

Pros

  • High performance: Zero-copy access to serialized data without parsing
  • Memory efficient: Minimal memory allocation during serialization and deserialization
  • Cross-platform: Supports multiple programming languages and platforms
  • Schema evolution: Allows for forward and backward compatibility

Cons

  • Learning curve: Requires understanding of the FlatBuffers schema language
  • Limited built-in types: May require custom serialization for complex data types
  • Larger file size: Serialized data can be larger compared to some alternatives
  • Less human-readable: Binary format is not easily readable without tools

Code Examples

Creating a FlatBuffer:

flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("John Doe");
auto person = CreatePerson(builder, name, 30);
builder.Finish(person);

Reading from a FlatBuffer:

auto person = GetPerson(buffer_pointer);
std::string name = person->name()->str();
int32_t age = person->age();

Mutating a FlatBuffer:

auto person = GetMutablePerson(mutable_buffer_pointer);
person->mutate_age(31);

Getting Started

  1. Install FlatBuffers:

    git clone https://github.com/google/flatbuffers.git
    cd flatbuffers
    cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release
    make
    
  2. Define your schema (e.g., person.fbs):

    table Person {
      name:string;
      age:int;
    }
    root_type Person;
    
  3. Compile the schema:

    ./flatc --cpp person.fbs
    
  4. Use the generated code in your C++ project:

    #include "person_generated.h"
    // Use FlatBuffers as shown in the code examples above
    

Competitor Comparisons

20,408

A modern formatting library

Pros of fmt

  • Focused on string formatting and output, providing a more intuitive and type-safe alternative to printf
  • Header-only library, making it easy to integrate into projects
  • Supports a wide range of formatting options and custom types

Cons of fmt

  • Limited to string formatting and output, not a general-purpose serialization solution
  • May have slightly higher compile times due to heavy template usage
  • Requires C++11 or later, which might be an issue for older codebases

Code Comparison

fmt:

#include <fmt/core.h>
std::string result = fmt::format("Hello, {}!", "world");
fmt::print("The answer is {}.", 42);

FlatBuffers:

#include "monster_generated.h"
flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("Orc");
auto monster = CreateMonster(builder, name, 0, 300);
builder.Finish(monster);

Summary

fmt is a powerful string formatting library, while FlatBuffers is a serialization system. They serve different purposes, with fmt excelling in text output and FlatBuffers focusing on efficient data serialization. Choose fmt for string manipulation and formatting, and FlatBuffers for cross-platform data exchange and storage.

65,113

Protocol Buffers - Google's data interchange format

Pros of Protocol Buffers

  • More mature and widely adopted, with extensive language support
  • Built-in support for backwards compatibility and schema evolution
  • Smaller message sizes for complex data structures with many optional fields

Cons of Protocol Buffers

  • Slower serialization and deserialization compared to FlatBuffers
  • Requires parsing the entire message before accessing any data
  • Larger binary size due to additional runtime library

Code Comparison

Protocol Buffers:

message Person {
  required string name = 1;
  optional int32 age = 2;
  repeated string hobbies = 3;
}

FlatBuffers:

table Person {
  name:string;
  age:int32;
  hobbies:[string];
}

Both examples define a simple Person structure with similar fields. Protocol Buffers uses keywords like required, optional, and repeated, while FlatBuffers uses a more straightforward table definition.

FlatBuffers offers zero-copy deserialization, allowing direct access to data without parsing the entire message. This results in faster performance for large datasets or when only partial data access is needed. However, Protocol Buffers provides better support for schema evolution and backwards compatibility, making it more suitable for systems with frequently changing data structures or long-term data storage requirements.

MessagePack implementation for C and C++ / msgpack.org[C/C++]

Pros of msgpack-c

  • Simpler and more lightweight serialization format
  • Wider language support and ecosystem
  • Faster serialization and deserialization for small data structures

Cons of msgpack-c

  • Less efficient for larger, more complex data structures
  • Lacks built-in schema evolution and backwards compatibility features
  • No direct support for zero-copy reads

Code Comparison

msgpack-c:

msgpack_pack_array(&pk, 3);
msgpack_pack_int(&pk, 1);
msgpack_pack_true(&pk);
msgpack_pack_str(&pk, 5);
msgpack_pack_str_body(&pk, "Hello", 5);

FlatBuffers:

auto name = builder.CreateString("Hello");
auto monster = CreateMonster(builder, 1, true, name);
builder.Finish(monster);

FlatBuffers requires a schema definition and generates code, while msgpack-c uses a more dynamic approach. FlatBuffers offers stronger typing and better performance for larger datasets, but msgpack-c is simpler for small, dynamic data structures.

Both libraries are widely used and have their strengths. FlatBuffers excels in scenarios requiring high performance and schema evolution, while msgpack-c is more suitable for simpler, lightweight serialization needs across multiple languages.

11,517

Cap'n Proto serialization/RPC system - core tools and C++ library

Pros of Cap'n Proto

  • Zero-copy deserialization, resulting in faster parsing
  • Built-in RPC system for distributed computing
  • More expressive schema language with advanced features like generics

Cons of Cap'n Proto

  • Larger message size due to padding and alignment requirements
  • Steeper learning curve and more complex API
  • Less widespread adoption compared to FlatBuffers

Code Comparison

Cap'n Proto schema:

struct Person {
  name @0 :Text;
  age @1 :UInt32;
  address @2 :Address;
}

FlatBuffers schema:

table Person {
  name:string;
  age:uint32;
  address:Address;
}

Both Cap'n Proto and FlatBuffers offer efficient serialization formats for structured data. Cap'n Proto excels in scenarios requiring zero-copy deserialization and distributed computing, while FlatBuffers provides a simpler API and smaller message sizes. The choice between them depends on specific project requirements, such as performance needs, message size constraints, and desired features like RPC support.

10,310

Apache Thrift

Pros of Thrift

  • Supports more programming languages than FlatBuffers
  • Provides built-in RPC (Remote Procedure Call) functionality
  • Offers a more mature ecosystem with broader adoption

Cons of Thrift

  • Generally slower serialization and deserialization compared to FlatBuffers
  • Larger message sizes, especially for small data structures
  • Requires full parsing of data before accessing any fields

Code Comparison

Thrift IDL:

struct Person {
  1: string name
  2: i32 age
  3: string email
}

FlatBuffers IDL:

table Person {
  name:string;
  age:int32;
  email:string;
}

Both IDLs define a simple Person structure with similar fields. The main difference is in syntax and how the fields are numbered (Thrift) or not (FlatBuffers).

FlatBuffers allows for zero-copy deserialization and direct access to fields without parsing the entire message, which can lead to performance benefits in certain scenarios. Thrift, on the other hand, offers more language support and built-in RPC capabilities, making it potentially more suitable for complex distributed systems.

42,154

JSON for Modern C++

Pros of json

  • Easier to read and write JSON data directly
  • Wider language support and ecosystem
  • More flexible for dynamic data structures

Cons of json

  • Slower parsing and serialization
  • Larger data size, especially for numeric values
  • Less type safety compared to strongly-typed schemas

Code Comparison

json:

#include <nlohmann/json.hpp>
using json = nlohmann::json;

json j = {
  {"name", "John"},
  {"age", 30},
  {"city", "New York"}
};
std::string serialized = j.dump();

FlatBuffers:

#include "person_generated.h"
flatbuffers::FlatBufferBuilder builder;
auto name = builder.CreateString("John");
auto person = CreatePerson(builder, name, 30, "New York");
builder.Finish(person);
uint8_t *buf = builder.GetBufferPointer();

FlatBuffers requires a schema definition and code generation, while json allows for more dynamic data manipulation. FlatBuffers offers better performance and smaller data size, but at the cost of flexibility and ease of use. The choice between the two depends on specific project requirements, such as performance needs, data structure complexity, and development workflow preferences.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

logo FlatBuffers

Build status BuildKite status Fuzzing Status OpenSSF Scorecard Join the chat at https://gitter.im/google/flatbuffers Discord Chat Twitter Follow Twitter Follow

FlatBuffers is a cross platform serialization library architected for maximum memory efficiency. It allows you to directly access serialized data without parsing/unpacking it first, while still having great forwards/backwards compatibility.

Quick Start

  1. Build the compiler for flatbuffers (flatc)

    Use cmake to create the build files for your platform and then perform the compilation (Linux example).

    cmake -G "Unix Makefiles"
    make -j
    
  2. Define your flatbuffer schema (.fbs)

    Write the schema to define the data you want to serialize. See monster.fbs for an example.

  3. Generate code for your language(s)

    Use the flatc compiler to take your schema and generate language-specific code:

    ./flatc --cpp --rust monster.fbs
    

    Which generates monster_generated.h and monster_generated.rs files.

  4. Serialize data

    Use the generated code, as well as the FlatBufferBuilder to construct your serialized buffer. (C++ example)

  5. Transmit/store/save Buffer

    Use your serialized buffer however you want. Send it to someone, save it for later, etc...

  6. Read the data

    Use the generated accessors to read the data from the serialized buffer.

    It doesn't need to be the same language/schema version, FlatBuffers ensures the data is readable across languages and schema versions. See the Rust example reading the data written by C++.

Documentation

Go to our landing page to browse our documentation.

Supported operating systems

  • Windows
  • macOS
  • Linux
  • Android
  • And any others with a recent C++ compiler (C++ 11 and newer)

Supported programming languages

Code generation and runtime libraries for many popular languages.

  1. C
  2. C++ - snapcraft.io
  3. C# - nuget.org
  4. Dart - pub.dev
  5. Go - go.dev
  6. Java - Maven
  7. JavaScript - NPM
  8. Kotlin
  9. Lobster
  10. Lua
  11. PHP
  12. Python - PyPI
  13. Rust - crates.io
  14. Swift - swiftpackageindex
  15. TypeScript - NPM
  16. Nim

Versioning

FlatBuffers does not follow traditional SemVer versioning (see rationale) but rather uses a format of the date of the release.

Contribution

To contribute to this project, see CONTRIBUTING.

Community

Security

Please see our Security Policy for reporting vulnerabilities.

Licensing

Flatbuffers is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.


NPM DownloadsLast 30 Days