Convert Figma logo to code with AI

google logoleveldb

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

36,378
7,805
36,378
330
View on GitHub

Top Related Projects

facebook's avatar

rocksdb

28,450

A library that provides an embeddable, persistent key-value store for fast storage.

apple's avatar

foundationdb

14,439

FoundationDB - the open source, distributed, transactional key-value store

syndtr's avatar

goleveldb

6,122

LevelDB key/value database in Go.

erthink's avatar

libmdbx

1,156

One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.

pmwkaa's avatar

sophia

1,853

Modern transactional key-value/row storage library.

Quick Overview

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values. It is designed to be lightweight and efficient, making it suitable for use in applications that require high-performance data storage and retrieval.

Pros

  • High performance with excellent read and write speeds
  • Supports atomic batch operations for improved consistency
  • Offers compression options to reduce storage space
  • Simple and easy-to-use API

Cons

  • Limited query capabilities compared to full-fledged databases
  • Lacks built-in support for concurrent access from multiple processes
  • No native support for Windows (though there are ports available)
  • Not suitable for applications requiring complex data structures or relationships

Code Examples

  1. Opening a database and writing a key-value pair:
#include <cassert>
#include "leveldb/db.h"

leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);
assert(status.ok());

status = db->Put(leveldb::WriteOptions(), "key1", "value1");
assert(status.ok());
  1. Reading a value from the database:
std::string value;
leveldb::Status s = db->Get(leveldb::ReadOptions(), "key1", &value);
if (s.ok()) {
    std::cout << "Read value: " << value << std::endl;
} else {
    std::cout << "Read failed: " << s.ToString() << std::endl;
}
  1. Iterating over key-value pairs:
leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions());
for (it->SeekToFirst(); it->Valid(); it->Next()) {
    std::cout << it->key().ToString() << ": " << it->value().ToString() << std::endl;
}
assert(it->status().ok());
delete it;

Getting Started

To use LevelDB in your C++ project:

  1. Clone the repository:

    git clone https://github.com/google/leveldb.git

  2. Build the library:

    cd leveldb
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release .. && cmake --build .

  3. Include the header files and link against the library in your project:

    #include "leveldb/db.h"
// Compile with: -I/path/to/leveldb/include -L/path/to/leveldb/build -lleveldb

  4. Use the LevelDB API in your code as shown in the examples above.

Competitor Comparisons

facebook logo

rocksdb

28,450

A library that provides an embeddable, persistent key-value store for fast storage.

Pros of RocksDB

  • Better performance for high write volumes and concurrent access
  • More features, including column families and transactions
  • Active development and frequent updates

Cons of RocksDB

  • Higher memory usage and larger disk footprint
  • More complex configuration and tuning required
  • Steeper learning curve due to additional features

Code Comparison

LevelDB:

leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);

RocksDB:

rocksdb::DB* db;
rocksdb::Options options;
options.create_if_missing = true;
rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db);

The basic usage of both databases is similar, with RocksDB being largely API-compatible with LevelDB. However, RocksDB offers additional options and features that can be configured for more advanced use cases.

RocksDB builds upon LevelDB's foundation, providing enhanced performance and additional features for high-volume, low-latency applications. While it offers more capabilities, it also requires more resources and expertise to optimize. The choice between the two depends on the specific requirements of your project, with RocksDB being better suited for more demanding applications that can benefit from its advanced features and performance optimizations.

apple logo

foundationdb

14,439

FoundationDB - the open source, distributed, transactional key-value store

Pros of FoundationDB

  • Distributed architecture for scalability and fault tolerance
  • ACID transactions across multiple keys and tables
  • Support for multiple data models (key-value, document, graph)

Cons of FoundationDB

  • Higher complexity and resource requirements
  • Steeper learning curve for implementation and management
  • Less suitable for simple, single-machine use cases

Code Comparison

FoundationDB (Python):

@fdb.transactional
def add_user(tr, user_id, name, email):
    tr[f'users:{user_id}:name'] = name
    tr[f'users:{user_id}:email'] = email

LevelDB (C++):

leveldb::Status s = db->Put(writeOptions, "user:1001:name", "John Doe");
s = db->Put(writeOptions, "user:1001:email", "john@example.com");

Key Differences

  • FoundationDB offers built-in transaction support, while LevelDB requires manual implementation
  • FoundationDB provides a distributed system out-of-the-box, whereas LevelDB is primarily designed for single-machine use
  • FoundationDB supports multiple data models, while LevelDB is strictly key-value

Use Case Recommendations

  • Choose FoundationDB for large-scale, distributed applications requiring strong consistency and complex data models
  • Opt for LevelDB in embedded systems, local applications, or when simplicity and low overhead are priorities
syndtr logo

goleveldb

6,122

LevelDB key/value database in Go.

Pros of goleveldb

  • Written in Go, providing native integration with Go projects
  • Implements the LevelDB API, allowing easy migration from C++ LevelDB
  • Active development and maintenance in recent years

Cons of goleveldb

  • May have slightly lower performance compared to the C++ implementation
  • Lacks some advanced features present in the original LevelDB
  • Smaller community and ecosystem compared to the original project

Code Comparison

LevelDB (C++):

#include <leveldb/db.h>

leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);

goleveldb (Go):

import "github.com/syndtr/goleveldb/leveldb"

db, err := leveldb.OpenFile("/tmp/testdb", nil)
if err != nil {
    // Handle error
}
defer db.Close()

Both implementations provide similar functionality, but goleveldb offers a more idiomatic Go API. The C++ version requires explicit options setting and error handling, while the Go version uses idiomatic error handling and default options. goleveldb simplifies the API for Go developers, making it easier to integrate into Go projects.

erthink logo

libmdbx

1,156

One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.

Pros of libmdbx

  • Better performance for read-heavy workloads due to its memory-mapped design
  • ACID compliance with full CRUD transactions
  • More flexible database size, supporting up to 16TB

Cons of libmdbx

  • Higher memory usage compared to LevelDB
  • Less suitable for write-heavy workloads
  • Steeper learning curve and more complex API

Code Comparison

LevelDB example:

leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);

libmdbx example:

MDBX_env *env;
mdbx_env_create(&env);
mdbx_env_set_maxdbs(env, 1);
mdbx_env_open(env, "/tmp/testdb", MDBX_NOSUBDIR, 0664);

Both libraries provide key-value storage, but libmdbx offers more advanced features like multi-version concurrency control (MVCC) and nested transactions. LevelDB is simpler to use and more suitable for write-intensive scenarios, while libmdbx excels in read-heavy environments and offers stronger consistency guarantees. The choice between them depends on specific project requirements and performance characteristics needed.

pmwkaa logo

sophia

1,853

Modern transactional key-value/row storage library.

Pros of Sophia

  • Designed for modern hardware with multi-core CPUs and SSDs
  • Supports ACID transactions and MVCC (Multi-Version Concurrency Control)
  • Offers better write amplification and space efficiency

Cons of Sophia

  • Less mature and less widely adopted compared to LevelDB
  • Smaller community and fewer third-party bindings
  • Documentation is not as comprehensive

Code Comparison

LevelDB:

leveldb::DB* db;
leveldb::Options options;
options.create_if_missing = true;
leveldb::Status status = leveldb::DB::Open(options, "/tmp/testdb", &db);

Sophia:

void *env = sp_env();
sp_setstring(env, "sophia.path", "/tmp/testdb", 0);
sp_setstring(env, "db", "test", 0);
int rc = sp_open(env);

Both LevelDB and Sophia are key-value storage engines, but they have different design philosophies and target use cases. LevelDB is more established and has a larger ecosystem, while Sophia aims to leverage modern hardware capabilities and offers additional features like ACID transactions. The choice between the two depends on specific project requirements, performance needs, and the importance of community support.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

This repository is receiving very limited maintenance. We will only review the following types of changes.

  • Fixes for critical bugs, such as data loss or memory corruption
  • Changes absolutely needed by internally supported leveldb clients. These typically fix breakage introduced by a language/standard library/OS update

ci

Authors: Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

Features

  • Keys and values are arbitrary byte arrays.
  • Data is stored sorted by key.
  • Callers can provide a custom comparison function to override the sort order.
  • The basic operations are Put(key,value), Get(key), Delete(key).
  • Multiple changes can be made in one atomic batch.
  • Users can create a transient snapshot to get a consistent view of data.
  • Forward and backward iteration is supported over the data.
  • Data is automatically compressed using the Snappy compression library, but Zstd compression is also supported.
  • External activity (file system operations etc.) is relayed through a virtual interface so users can customize the operating system interactions.

Documentation

LevelDB library documentation is online and bundled with the source code.

Limitations

  • This is not a SQL database. It does not have a relational data model, it does not support SQL queries, and it has no support for indexes.
  • Only a single process (possibly multi-threaded) can access a particular database at a time.
  • There is no client-server support builtin to the library. An application that needs such support will have to wrap their own server around the library.

Getting the Source

git clone --recurse-submodules https://github.com/google/leveldb.git

Building

This project supports CMake out of the box.

Build for POSIX

Quick start:

mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release .. && cmake --build .

Building for Windows

First generate the Visual Studio 2017 project/solution files:

mkdir build
cd build
cmake -G "Visual Studio 15" ..

The default default will build for x86. For 64-bit run:

cmake -G "Visual Studio 15 Win64" ..

To compile the Windows solution from the command-line:

devenv /build Debug leveldb.sln

or open leveldb.sln in Visual Studio and build from within.

Please see the CMake documentation and CMakeLists.txt for more advanced usage.

Contributing to the leveldb Project

This repository is receiving very limited maintenance. We will only review the following types of changes.

  • Bug fixes
  • Changes absolutely needed by internally supported leveldb clients. These typically fix breakage introduced by a language/standard library/OS update

The leveldb project welcomes contributions. leveldb's primary goal is to be a reliable and fast key/value store. Changes that are in line with the features/limitations outlined above, and meet the requirements below, will be considered.

Contribution requirements:

  1. Tested platforms only. We generally will only accept changes for platforms that are compiled and tested. This means POSIX (for Linux and macOS) or Windows. Very small changes will sometimes be accepted, but consider that more of an exception than the rule.

  2. Stable API. We strive very hard to maintain a stable API. Changes that require changes for projects using leveldb might be rejected without sufficient benefit to the project.

  3. Tests: All changes must be accompanied by a new (or changed) test, or a sufficient explanation as to why a new (or changed) test is not required.

  4. Consistent Style: This project conforms to the Google C++ Style Guide. To ensure your changes are properly formatted please run:

    clang-format -i --style=file <file>

We are unlikely to accept contributions to the build configuration files, such as CMakeLists.txt. We are focused on maintaining a build configuration that allows us to test that the project works in a few supported configurations inside Google. We are not currently interested in supporting other requirements, such as different operating systems, compilers, or build systems.

Submitting a Pull Request

Before any pull request will be accepted the author must first sign a Contributor License Agreement (CLA) at https://cla.developers.google.com/.

In order to keep the commit timeline linear squash your changes down to a single commit and rebase on google/leveldb/main. This keeps the commit timeline linear and more easily sync'ed with the internal repository at Google. More information at GitHub's About Git rebase page.

Performance

Here is a performance report (with explanations) from the run of the included db_bench program. The results are somewhat noisy, but should be enough to get a ballpark performance estimate.

Setup

We use a database with a million entries. Each entry has a 16 byte key, and a 100 byte value. Values used by the benchmark compress to about half their original size.

LevelDB:    version 1.1
Date:       Sun May  1 12:11:26 2011
CPU:        4 x Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
CPUCache:   4096 KB
Keys:       16 bytes each
Values:     100 bytes each (50 bytes after compression)
Entries:    1000000
Raw Size:   110.6 MB (estimated)
File Size:  62.9 MB (estimated)

Write performance

The "fill" benchmarks create a brand new database, in either sequential, or random order. The "fillsync" benchmark flushes data from the operating system to the disk after every operation; the other write operations leave the data sitting in the operating system buffer cache for a while. The "overwrite" benchmark does random writes that update existing keys in the database.

fillseq      :       1.765 micros/op;   62.7 MB/s
fillsync     :     268.409 micros/op;    0.4 MB/s (10000 ops)
fillrandom   :       2.460 micros/op;   45.0 MB/s
overwrite    :       2.380 micros/op;   46.5 MB/s

Each "op" above corresponds to a write of a single key/value pair. I.e., a random write benchmark goes at approximately 400,000 writes per second.

Each "fillsync" operation costs much less (0.3 millisecond) than a disk seek (typically 10 milliseconds). We suspect that this is because the hard disk itself is buffering the update in its memory and responding before the data has been written to the platter. This may or may not be safe based on whether or not the hard disk has enough power to save its memory in the event of a power failure.

Read performance

We list the performance of reading sequentially in both the forward and reverse direction, and also the performance of a random lookup. Note that the database created by the benchmark is quite small. Therefore the report characterizes the performance of leveldb when the working set fits in memory. The cost of reading a piece of data that is not present in the operating system buffer cache will be dominated by the one or two disk seeks needed to fetch the data from disk. Write performance will be mostly unaffected by whether or not the working set fits in memory.

readrandom  : 16.677 micros/op;  (approximately 60,000 reads per second)
readseq     :  0.476 micros/op;  232.3 MB/s
readreverse :  0.724 micros/op;  152.9 MB/s

LevelDB compacts its underlying storage data in the background to improve read performance. The results listed above were done immediately after a lot of random writes. The results after compactions (which are usually triggered automatically) are better.

readrandom  : 11.602 micros/op;  (approximately 85,000 reads per second)
readseq     :  0.423 micros/op;  261.8 MB/s
readreverse :  0.663 micros/op;  166.9 MB/s

Some of the high cost of reads comes from repeated decompression of blocks read from disk. If we supply enough cache to the leveldb so it can hold the uncompressed blocks in memory, the read performance improves again:

readrandom  : 9.775 micros/op;  (approximately 100,000 reads per second before compaction)
readrandom  : 5.215 micros/op;  (approximately 190,000 reads per second after compaction)

Repository contents

See doc/index.md for more explanation. See doc/impl.md for a brief overview of the implementation.

The public interface is in include/leveldb/*.h. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Guide to header files:

  • include/leveldb/db.h: Main interface to the DB: Start here.

  • include/leveldb/options.h: Control over the behavior of an entire database, and also control over the behavior of individual reads and writes.

  • include/leveldb/comparator.h: Abstraction for user-specified comparison function. If you want just bytewise comparison of keys, you can use the default comparator, but clients can write their own comparator implementations if they want custom ordering (e.g. to handle different character encodings, etc.).

  • include/leveldb/iterator.h: Interface for iterating over data. You can get an iterator from a DB object.

  • include/leveldb/write_batch.h: Interface for atomically applying multiple updates to a database.

  • include/leveldb/slice.h: A simple module for maintaining a pointer and a length into some other byte array.

  • include/leveldb/status.h: Status is returned from many of the public interfaces and is used to report success and various kinds of errors.

  • include/leveldb/env.h: Abstraction of the OS environment. A posix implementation of this interface is in util/env_posix.cc.

  • include/leveldb/table.h, include/leveldb/table_builder.h: Lower-level modules that most clients probably won't use directly.