Top Related Projects
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Protocol Buffers - Google's data interchange format
Apache Avro is a data serialization system.
MessagePack is an extremely efficient object serialization library. It's like JSON, but very fast and small.
Main Portal page for the Jackson project
Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
Quick Overview
Apache Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between various programming languages. Thrift is designed to be efficient and reliable, making it suitable for large-scale distributed systems.
Pros
- Supports multiple programming languages, enabling seamless communication between different technology stacks
- Provides efficient binary serialization, resulting in better performance compared to text-based protocols
- Offers both synchronous and asynchronous communication models
- Includes built-in versioning support, allowing for easier service evolution
Cons
- Steeper learning curve compared to simpler alternatives like REST or gRPC
- Limited support for some less common programming languages
- Can be complex to set up and configure, especially for smaller projects
- Documentation can be inconsistent or outdated for some language implementations
Code Examples
- Defining a Thrift service:
namespace cpp calculator
namespace py calculator
service Calculator {
i32 add(1:i32 num1, 2:i32 num2),
i32 subtract(1:i32 num1, 2:i32 num2),
i32 multiply(1:i32 num1, 2:i32 num2),
double divide(1:i32 num1, 2:i32 num2) throws (1:ArithmeticException ae)
}
This example defines a simple Calculator service with four methods.
- Implementing the service in Python:
from calculator import Calculator
class CalculatorHandler:
def add(self, num1, num2):
return num1 + num2
def subtract(self, num1, num2):
return num1 - num2
def multiply(self, num1, num2):
return num1 * num2
def divide(self, num1, num2):
if num2 == 0:
raise ArithmeticException("Division by zero")
return num1 / num2
handler = CalculatorHandler()
processor = Calculator.Processor(handler)
This code implements the Calculator service in Python.
- Creating a Thrift client in C++:
#include <thrift/protocol/TBinaryProtocol.h>
#include <thrift/transport/TSocket.h>
#include <thrift/transport/TTransportUtils.h>
#include "Calculator.h"
using namespace apache::thrift;
using namespace apache::thrift::protocol;
using namespace apache::thrift::transport;
int main() {
std::shared_ptr<TTransport> socket(new TSocket("localhost", 9090));
std::shared_ptr<TTransport> transport(new TBufferedTransport(socket));
std::shared_ptr<TProtocol> protocol(new TBinaryProtocol(transport));
CalculatorClient client(protocol);
try {
transport->open();
int result = client.add(5, 7);
std::cout << "5 + 7 = " << result << std::endl;
transport->close();
} catch (TException& tx) {
std::cout << "ERROR: " << tx.what() << std::endl;
}
return 0;
}
This example shows how to create a Thrift client in C++ to call the add
method of the Calculator service.
Getting Started
-
Install Thrift compiler:
sudo apt-get install thrift-compiler
-
Define your Thrift service in a
.thrift
file. -
Generate code for your target languages:
thrift --gen cpp calculator.thrift thrift --gen py calculator.thrift
-
Implement the service in your chosen language.
-
Set up a server to host the service and create clients to interact with it.
Competitor Comparisons
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Pros of gRPC
- Better performance and lower latency due to HTTP/2 and Protocol Buffers
- Stronger support for bi-directional streaming and server-side streaming
- More extensive language support and active community development
Cons of gRPC
- Steeper learning curve, especially for developers new to Protocol Buffers
- Less flexibility in data serialization formats compared to Thrift
- Limited browser support without additional tools or proxies
Code Comparison
gRPC (Protocol Buffers):
syntax = "proto3";
service Greeter {
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}
Thrift:
service Greeter {
string sayHello(1: string name)
}
struct HelloReply {
1: string message
}
Both gRPC and Thrift are popular RPC frameworks, but they have different strengths. gRPC excels in performance and streaming capabilities, while Thrift offers more flexibility in data serialization. The choice between them often depends on specific project requirements and the development team's familiarity with each technology.
Protocol Buffers - Google's data interchange format
Pros of Protocol Buffers
- More compact serialization, resulting in smaller message sizes
- Better performance in terms of serialization and deserialization speed
- Stronger type checking and better support for schema evolution
Cons of Protocol Buffers
- Limited language support compared to Thrift
- Less flexible in terms of transport protocols and server implementations
- Steeper learning curve for developers new to the technology
Code Comparison
Protocol Buffers:
syntax = "proto3";
message Person {
string name = 1;
int32 age = 2;
}
Thrift:
struct Person {
1: string name
2: i32 age
}
Both Protocol Buffers and Thrift are popular serialization frameworks used for efficient data exchange between services. Protocol Buffers offers better performance and more compact serialization, while Thrift provides broader language support and more flexibility in terms of transport protocols.
The code comparison shows that both frameworks use similar syntax for defining data structures, with minor differences in keywords and type declarations. Protocol Buffers uses the message
keyword, while Thrift uses struct
. Type declarations also differ slightly, with Protocol Buffers using int32
and Thrift using i32
for 32-bit integers.
Ultimately, the choice between Protocol Buffers and Thrift depends on specific project requirements, such as performance needs, language support, and team expertise.
Apache Avro is a data serialization system.
Pros of Avro
- Better schema evolution support, allowing easier updates to data structures
- More compact serialization format, resulting in smaller data sizes
- Built-in support for data compression
Cons of Avro
- Less language support compared to Thrift
- Steeper learning curve for developers new to the ecosystem
Code Comparison
Avro schema definition:
{
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "age", "type": "int"}
]
}
Thrift schema definition:
struct User {
1: string name
2: i32 age
}
Both Avro and Thrift provide efficient serialization and deserialization of structured data. Avro offers better schema evolution and more compact serialization, while Thrift has broader language support. The choice between the two depends on specific project requirements, such as data size constraints, language ecosystem, and the need for schema flexibility.
MessagePack is an extremely efficient object serialization library. It's like JSON, but very fast and small.
Pros of MessagePack
- Simpler and more lightweight serialization format
- Faster serialization and deserialization performance
- Wider language support and easier integration
Cons of MessagePack
- Lacks built-in RPC framework
- Less robust type system compared to Thrift
- Limited support for schema evolution
Code Comparison
MessagePack:
import msgpack
data = {"name": "John", "age": 30}
packed = msgpack.packb(data)
unpacked = msgpack.unpackb(packed)
Thrift:
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from person_types import Person
person = Person(name="John", age=30)
transport = TTransport.TMemoryBuffer()
protocol = TBinaryProtocol.TBinaryProtocol(transport)
person.write(protocol)
serialized = transport.getvalue()
MessagePack offers a more straightforward API for serialization and deserialization, while Thrift requires more setup and boilerplate code. However, Thrift provides stronger typing and a more comprehensive RPC framework, which can be beneficial for larger, more complex systems. MessagePack is often preferred for simpler use cases and when performance is a primary concern, while Thrift is better suited for enterprise-level applications with complex data structures and service definitions.
Main Portal page for the Jackson project
Pros of Jackson
- More flexible and supports a wider range of data formats (JSON, XML, YAML, etc.)
- Better performance for JSON serialization/deserialization
- More active development and community support
Cons of Jackson
- Steeper learning curve due to more extensive API
- Larger library size, which may impact application size
- Less focus on cross-language support compared to Thrift
Code Comparison
Jackson:
ObjectMapper mapper = new ObjectMapper();
MyObject obj = mapper.readValue(jsonString, MyObject.class);
String json = mapper.writeValueAsString(obj);
Thrift:
TSerializer serializer = new TSerializer();
byte[] bytes = serializer.serialize(thriftObject);
TDeserializer deserializer = new TDeserializer();
deserializer.deserialize(thriftObject, bytes);
Key Differences
- Jackson primarily focuses on JSON processing, while Thrift is designed for cross-language RPC and data serialization
- Thrift generates code for multiple languages, whereas Jackson works within the Java ecosystem
- Jackson offers more flexibility in handling various data formats, while Thrift provides a more structured approach to data definition and serialization
Use Cases
- Choose Jackson for Java-based projects requiring flexible JSON processing
- Opt for Thrift when building cross-language services or need a strict data contract between systems
Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
Pros of Arrow
- Designed for efficient in-memory processing and zero-copy data sharing
- Supports a wider range of data types and complex nested structures
- Offers better performance for analytical workloads and big data processing
Cons of Arrow
- Steeper learning curve due to its more complex architecture
- Less suitable for simple RPC use cases compared to Thrift
- Requires more memory for data representation in some cases
Code Comparison
Arrow (Python):
import pyarrow as pa
data = [
{'name': 'John', 'age': 30},
{'name': 'Jane', 'age': 25}
]
table = pa.Table.from_pylist(data)
Thrift (Python):
from person_types import Person
person = Person()
person.name = "John"
person.age = 30
Key Differences
- Arrow focuses on columnar data representation, while Thrift is primarily for RPC and serialization
- Arrow provides a memory-efficient data format, whereas Thrift emphasizes cross-language compatibility
- Arrow is better suited for data analytics and processing large datasets, while Thrift excels in service-to-service communication
Use Cases
- Arrow: Big data processing, analytics, and machine learning pipelines
- Thrift: Microservices communication, cross-language RPC, and simple data serialization
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Apache Thrift
Introduction
Thrift is a lightweight, language-independent software stack for point-to-point RPC implementation. Thrift provides clean abstractions and implementations for data transport, data serialization, and application level processing. The code generation system takes a simple definition language as input and generates code across programming languages that uses the abstracted stack to build interoperable RPC clients and servers.
Thrift makes it easy for programs written in different programming languages to share data and call remote procedures. With support for 28 programming languages, chances are Thrift supports the languages that you currently use.
Thrift is specifically designed to support non-atomic version changes across client and server code. This allows you to upgrade your server while still being able to service older clients; or have newer clients issue requests to older servers. An excellent community-provided write-up about thrift and compatibility when versioning an API can be found in the Thrift Missing Guide.
For more details on Thrift's design and implementation, see the Thrift whitepaper included in this distribution, or at the README.md file in your particular subdirectory of interest.
Status
Branch | Travis | Appveyor | Coverity Scan | codecov.io | Website |
---|---|---|---|---|---|
master | |||||
0.17.0 |
Releases
Thrift does not maintain a specific release calendar at this time.
We strive to release twice yearly. Download the current release.
License
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Project Hierarchy
thrift/
compiler/
Contains the Thrift compiler, implemented in C++.
lib/
Contains the Thrift software library implementation, subdivided by
language of implementation.
cpp/
go/
java/
php/
py/
rb/
...
test/
Contains sample Thrift files and test code across the target programming
languages.
tutorial/
Contains a basic tutorial that will teach you how to develop software
using Thrift.
Development
To build the same way Travis CI builds the project you should use docker. We have comprehensive building instructions for docker.
Requirements
See http://thrift.apache.org/docs/install for a list of build requirements (may be stale). Alternatively, see the docker build environments for a list of prerequisites.
Resources
More information about Thrift can be obtained on the Thrift webpage at:
http://thrift.apache.org
Acknowledgments
Thrift was inspired by pillar, a lightweight RPC tool written by Adam D'Angelo, and also by Google's protocol buffers.
Installation
If you are building from the first time out of the source repository, you will need to generate the configure scripts. (This is not necessary if you downloaded a tarball.) From the top directory, do:
./bootstrap.sh
Once the configure scripts are generated, thrift can be configured. From the top directory, do:
./configure
You may need to specify the location of the boost files explicitly.
If you installed boost in /usr/local
, you would run configure as follows:
./configure --with-boost=/usr/local
Note that by default the thrift C++ library is typically built with debugging symbols included. If you want to customize these options you should use the CXXFLAGS option in configure, as such:
./configure CXXFLAGS='-g -O2'
./configure CFLAGS='-g -O2'
./configure CPPFLAGS='-DDEBUG_MY_FEATURE'
To enable gcov required options -fprofile-arcs -ftest-coverage enable them:
./configure --enable-coverage
Run ./configure --help to see other configuration options
Please be aware that the Python library will ignore the --prefix option
and just install wherever Python's distutils puts it (usually along
the lines of /usr/lib/pythonX.Y/site-packages/
). If you need to control
where the Python modules are installed, set the PY_PREFIX variable.
(DESTDIR is respected for Python and C++.)
Make thrift:
make
From the top directory, become superuser and do:
make install
Uninstall thrift:
make uninstall
Note that some language packages must be installed manually using build tools better suited to those languages (at the time of this writing, this applies to Java, Ruby, PHP).
Look for the README.md file in the lib/
Package Managers
Apache Thrift is available via a number of package managers, a list which is is steadily growing. A more detailed overview can be found at the Apache Thrift web site under "Libraries" and/or in the respective READMEs for each language under /lib
Testing
There are a large number of client library tests that can all be run from the top-level directory.
make -k check
This will make all of the libraries (as necessary), and run through the unit tests defined in each of the client libraries. If a single language fails, the make check will continue on and provide a synopsis at the end.
To run the cross-language test suite, please run:
make cross
This will run a set of tests that use different language clients and servers.
Top Related Projects
The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)
Protocol Buffers - Google's data interchange format
Apache Avro is a data serialization system.
MessagePack is an extremely efficient object serialization library. It's like JSON, but very fast and small.
Main Portal page for the Jackson project
Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot