foundationdb
FoundationDB - the open source, distributed, transactional key-value store
Top Related Projects
CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
Distributed reliable key-value store for the most critical data of a distributed system
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
The high-performance database for modern applications
Apache Cassandra®
Quick Overview
FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It offers a robust, scalable, and transactional key-value store that combines NoSQL's scalability with the ACID guarantees of traditional databases.
Pros
- High scalability and performance for distributed systems
- Strong consistency and ACID transactions
- Multi-model support (key-value, document, graph, etc.)
- Language-agnostic with bindings for multiple programming languages
Cons
- Steep learning curve for newcomers to distributed systems
- Limited built-in support for complex queries compared to traditional SQL databases
- Requires careful configuration and tuning for optimal performance
- Smaller community compared to some other popular databases
Code Examples
- Basic key-value operations:
import fdb
fdb.api_version(630)
db = fdb.open()
@fdb.transactional
def set_value(tr, key, value):
tr[key] = value
@fdb.transactional
def get_value(tr, key):
return tr[key]
set_value(db, b'hello', b'world')
result = get_value(db, b'hello')
print(result) # Output: b'world'
- Using atomic operations:
import fdb
fdb.api_version(630)
db = fdb.open()
@fdb.transactional
def increment_counter(tr, key):
tr.add(key, struct.pack('<q', 1))
@fdb.transactional
def get_counter(tr, key):
return struct.unpack('<q', tr[key])[0]
increment_counter(db, b'counter')
increment_counter(db, b'counter')
count = get_counter(db, b'counter')
print(count) # Output: 2
- Working with directories:
import fdb
import fdb.directory
fdb.api_version(630)
db = fdb.open()
@fdb.transactional
def create_and_use_directory(tr):
dir = fdb.directory.create_or_open(tr, ('my_app', 'users'))
dir['alice'] = b'data for alice'
return dir['alice']
result = create_and_use_directory(db)
print(result) # Output: b'data for alice'
Getting Started
- Install FoundationDB server and client libraries for your platform.
- Install the Python client:
pip install foundationdb
- Create a basic script:
import fdb fdb.api_version(630) db = fdb.open() @fdb.transactional def hello_world(tr): tr[b'hello'] = b'world' return tr[b'hello'] result = hello_world(db) print(result) # Output: b'world'
- Run the script to interact with FoundationDB.
Competitor Comparisons
CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
Pros of CockroachDB
- Designed for global scale and geographic distribution
- SQL-compatible with strong consistency guarantees
- Built-in multi-active availability for high resilience
Cons of CockroachDB
- Higher resource consumption and operational complexity
- Potentially slower performance for certain workloads
- Steeper learning curve for optimization and tuning
Code Comparison
CockroachDB (SQL-based):
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name STRING,
created_at TIMESTAMP DEFAULT current_timestamp()
);
FoundationDB (Key-Value based):
@fdb.transactional
def create_user(tr, name):
user_id = uuid.uuid4()
tr[f'users/{user_id}/name'] = name
tr[f'users/{user_id}/created_at'] = time.time()
return user_id
CockroachDB offers a familiar SQL interface, making it easier for developers with SQL experience. FoundationDB provides a low-level key-value API, offering more flexibility but requiring more custom logic for data modeling.
Both databases focus on scalability and consistency, but CockroachDB is specifically designed for distributed SQL workloads, while FoundationDB serves as a foundation for building various distributed systems.
Distributed reliable key-value store for the most critical data of a distributed system
Pros of etcd
- Simpler architecture and easier to set up for small to medium-scale distributed systems
- Built-in support for distributed consensus using the Raft algorithm
- Widely adopted in the Kubernetes ecosystem, making it a natural choice for cloud-native applications
Cons of etcd
- Limited scalability compared to FoundationDB, especially for large-scale deployments
- Less flexible data model, primarily focused on key-value storage
- Lower overall performance in terms of throughput and latency for complex operations
Code Comparison
etcd (Go):
cli, _ := clientv3.New(clientv3.Config{Endpoints: []string{"localhost:2379"}})
defer cli.Close()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
_, err := cli.Put(ctx, "key", "value")
cancel()
FoundationDB (C++):
Database db = Database::createDatabase("fdb.cluster");
Transaction tr(db);
tr.set(KeyRef("key"), ValueRef("value"));
tr.commit().wait();
Both examples demonstrate basic key-value operations, but FoundationDB's API is more low-level and offers finer control over transactions.
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
Pros of TiDB
- Designed for distributed HTAP (Hybrid Transactional/Analytical Processing) workloads
- SQL-compatible and MySQL protocol support, easier migration for existing MySQL users
- Built-in horizontal scalability and high availability features
Cons of TiDB
- Higher complexity due to multiple components (TiKV, PD, TiDB)
- Potentially higher resource requirements for small-scale deployments
- Steeper learning curve for administration and optimization
Code Comparison
TiDB (SQL-based):
CREATE TABLE users (
id INT PRIMARY KEY,
name VARCHAR(50),
age INT
);
INSERT INTO users VALUES (1, 'Alice', 30);
SELECT * FROM users WHERE age > 25;
FoundationDB (Key-Value based):
@fdb.transactional
def create_user(tr, user_id, name, age):
tr[f'users:{user_id}:name'] = name
tr[f'users:{user_id}:age'] = str(age)
@fdb.transactional
def get_users_over_25(tr):
return [k.split(':')[1] for k, v in tr.get_range_startswith('users:')
if k.endswith(':age') and int(v) > 25]
FoundationDB offers a low-level key-value API, while TiDB provides a SQL interface. FoundationDB requires more custom code for data modeling and querying, but offers fine-grained control over data storage and retrieval. TiDB's SQL interface is more familiar and easier to use for traditional database users, but may have less flexibility for certain use cases.
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
Pros of ArangoDB
- Multi-model database supporting key/value, document, and graph data models
- Flexible query language (AQL) for complex data operations
- Built-in web interface for administration and data exploration
Cons of ArangoDB
- Higher memory usage compared to FoundationDB
- Less focus on extreme scalability and distributed systems
- Steeper learning curve due to multiple data models and query language
Code Comparison
FoundationDB (C++):
Transaction tr(db);
tr.set(KeyRef("hello"), ValueRef("world"));
tr.commit().wait();
ArangoDB (JavaScript):
db.collection('myCollection').save({ hello: 'world' });
Key Differences
- FoundationDB is a distributed key-value store, while ArangoDB is a multi-model database
- FoundationDB emphasizes scalability and ACID transactions, ArangoDB focuses on flexibility
- FoundationDB uses a lower-level API, ArangoDB provides higher-level abstractions
- FoundationDB is better suited for large-scale distributed systems, ArangoDB for diverse data modeling needs
Both databases have their strengths and are suitable for different use cases. FoundationDB excels in highly scalable, transactional environments, while ArangoDB offers more versatility in data modeling and querying.
The high-performance database for modern applications
Pros of Dgraph
- Native support for GraphQL, making it easier to work with graph-based data models
- Built-in support for distributed transactions and horizontal scaling
- Designed specifically for graph data, offering optimized performance for graph queries
Cons of Dgraph
- Less mature and battle-tested compared to FoundationDB
- Smaller community and ecosystem
- More specialized use case, primarily focused on graph databases
Code Comparison
FoundationDB (Key-Value Store):
@fdb.transactional
def add_user(tr, user_id, name):
tr[f"users:{user_id}:name"] = name
fdb.open()
add_user(db, "123", "John Doe")
Dgraph (GraphQL):
mutation {
addUser(input: [{id: "123", name: "John Doe"}]) {
user {
id
name
}
}
}
Summary
FoundationDB is a more general-purpose distributed database with a key-value store model, while Dgraph is specifically designed for graph data and offers native GraphQL support. FoundationDB has a larger community and is more battle-tested, but Dgraph provides optimized performance for graph-based queries and built-in support for distributed transactions. The choice between the two depends on the specific use case and data model requirements of the project.
Apache Cassandra®
Pros of Cassandra
- Highly scalable and distributed architecture, designed for large-scale deployments
- Strong support for multi-datacenter replication and high availability
- Flexible data model with support for wide-column storage
Cons of Cassandra
- Complex setup and maintenance, requiring significant operational expertise
- Limited support for ACID transactions compared to FoundationDB
- Can be resource-intensive, especially for smaller deployments
Code Comparison
Cassandra (CQL):
CREATE TABLE users (
id UUID PRIMARY KEY,
name TEXT,
email TEXT
);
FoundationDB (Python API):
@fdb.transactional
def create_user(tr, id, name, email):
tr[f'users/{id}/name'] = name
tr[f'users/{id}/email'] = email
FoundationDB uses a key-value approach, while Cassandra provides a more structured table-based model. FoundationDB's transactional nature is evident in the code example, showcasing its strong consistency guarantees.
Both databases have their strengths, with Cassandra excelling in large-scale distributed scenarios and FoundationDB offering robust ACID compliance. The choice between them depends on specific use cases, scalability requirements, and consistency needs.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations. It is especially well-suited for read/write workloads but also has excellent performance for write-intensive workloads. Users interact with the database using API language binding.
To learn more about FoundationDB, visit foundationdb.org
Documentation
Documentation can be found online at https://apple.github.io/foundationdb/. The documentation covers details of API usage, background information on design philosophy, and extensive usage examples. Docs are built from the source in this repo.
Forums
The FoundationDB Forums are the home for most of the discussion and communication about the FoundationDB project. We welcome your participation! We want FoundationDB to be a great project to be a part of and, as part of that, have established a Code of Conduct to establish what constitutes permissible modes of interaction.
Contributing
Contributing to FoundationDB can be in contributions to the code base, sharing your experience and insights in the community on the Forums, or contributing to projects that make use of FoundationDB. Please see the contributing guide for more specifics.
Getting Started
Binary downloads
Developers interested in using FoundationDB can get started by downloading and installing a binary package. Please see the downloads page for a list of available packages.
Compiling from source
Developers on an OS for which there is no binary package, or who would like to start hacking on the code, can get started by compiling from source.
The official docker image for building is foundationdb/build
, which has all dependencies installed. The Docker image definitions used by FoundationDB team members can be found in the dedicated repository.
To build outside of the official docker image, you'll need at least these dependencies:
If compiling for local development, please set -DUSE_WERROR=ON
in
cmake. Our CI compiles with -Werror
on, so this way you'll find out about
compiler warnings that break the build earlier.
Once you have your dependencies, you can run cmake and then build:
- Check out this repository.
- Create a build directory (you can have the build directory anywhere you like).
cd <PATH_TO_BUILD_DIRECTORY>
cmake -G Ninja <PATH_TO_FOUNDATIONDB_DIRECTORY>
ninja # If this crashes it probably ran out of memory. Try ninja -j1
Language Bindings
The language bindings that are supported by cmake will have a corresponding
README.md
file in the corresponding bindings/lang
directory.
Generally, cmake will build all language bindings for which it can find all necessary dependencies. After each successful cmake run, cmake will tell you which language bindings it is going to build.
Generating compile_commands.json
CMake can build a compilation database for you. However, the default generated
one is not too useful as it operates on the generated files. When running make,
the build system will create another compile_commands.json
file in the source
directory. This can than be used for tools like
CCLS,
CQuery, etc. This way you can get
code-completion and code navigation in flow. It is not yet perfect (it will show
a few errors) but we are constantly working on improving the development experience.
CMake will not produce a compile_commands.json
, you must pass
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON
. This also enables the target
processed_compile_commands
, which rewrites compile_commands.json
to
describe the actor compiler source file, not the post-processed output files,
and places the output file in the source directory. This file should then be
picked up automatically by any tooling.
Note that if building inside of the foundationdb/build
docker
image, the resulting paths will still be incorrect and require manual fixing.
One will wish to re-run cmake
with -DCMAKE_EXPORT_COMPILE_COMMANDS=OFF
to
prevent it from reverting the manual changes.
Using IDEs
CMake has built in support for a number of popular IDEs. However, because flow files are precompiled with the actor compiler, an IDE will not be very useful as a user will only be presented with the generated code - which is not what she wants to edit and get IDE features for.
The good news is, that it is possible to generate project files for editing
flow with a supported IDE. There is a CMake option called OPEN_FOR_IDE
which
will generate a project which can be opened in an IDE for editing. You won't be
able to build this project, but you will be able to edit the files and get most
edit and navigation features your IDE supports.
For example, if you want to use Xcode to make changes to FoundationDB you can create an Xcode project with the following command:
cmake -G Xcode -DOPEN_FOR_IDE=ON <FDB_SOURCE_DIRECTORY>
You should create a second build-directory which you will use for building and debugging.
FreeBSD
-
Check out this repo on your server.
-
Install compile-time dependencies from ports.
-
(Optional) Use tmpfs & ccache for significantly faster repeat builds
-
(Optional) Install a JDK for Java Bindings. FoundationDB currently builds with Java 8.
-
Navigate to the directory where you checked out the foundationdb repo.
-
Build from source.
sudo pkg install -r FreeBSD \ shells/bash devel/cmake devel/ninja devel/ccache \ lang/mono lang/python3 \ devel/boost-libs devel/libeio \ security/openssl mkdir .build && cd .build cmake -G Ninja \ -DUSE_CCACHE=on \ -DUSE_DTRACE=off \ .. ninja -j 10 # run fast tests ctest -L fast # run all tests ctest --output-on-failure -v
Linux
There are no special requirements for Linux. A docker image can be pulled from
foundationdb/build
that has all of FoundationDB's dependencies
pre-installed, and is what the CI uses to build and test PRs.
cmake -G Ninja <FDB_SOURCE_DIR>
ninja
cpack -G DEB
For RPM simply replace DEB
with RPM
.
MacOS
The build under MacOS will work the same way as on Linux. To get boost and ninja you can use Homebrew.
cmake -G Ninja <PATH_TO_FOUNDATIONDB_SOURCE>
To generate a installable package,
ninja
$SRCDIR/packaging/osx/buildpkg.sh . $SRCDIR
Windows
Under Windows, only Visual Studio with ClangCl is supported
- Install Visual Studio 2019 (IDE or Build Tools), and enable llvm support
- Install CMake 3.15 or higher
- Download Boost 1.77.0
- Unpack boost to C:\boost, or use
-DBOOST_ROOT=<PATH_TO_BOOST>
withcmake
if unpacked elsewhere - Install Python if is not already installed by Visual Studio
- (Optional) Install OpenJDK 11 to build Java bindings
- (Optional) Install OpenSSL 3.x to build with TLS support
- (Optional) Install WIX Toolset to build Windows installer
mkdir build && cd build
cmake -G "Visual Studio 16 2019" -A x64 -T ClangCl <PATH_TO_FOUNDATIONDB_SOURCE>
msbuild /p:Configuration=Release foundationdb.sln
- To increase build performance, use
/p:UseMultiToolTask=true
and/p:CL_MPCount=<NUMBER_OF_PARALLEL_JOBS>
Top Related Projects
CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
Distributed reliable key-value store for the most critical data of a distributed system
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
The high-performance database for modern applications
Apache Cassandra®
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot