Convert Figma logo to code with AI

tikv logotikv

Distributed transactional key-value database, originally created to complement TiDB

15,644
2,169
15,644
1,517

Top Related Projects

37,777

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.

30,408

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

5,271

Apache HBase

Apache Cassandra®

48,227

Distributed reliable key-value store for the most critical data of a distributed system

14,262

NoSQL data store using the Seastar framework, compatible with Apache Cassandra and Amazon DynamoDB

Quick Overview

TiKV is a distributed transactional key-value database built in Rust. It is designed to support large-scale applications with high performance, scalability, and reliability. TiKV is the storage layer for TiDB, a distributed NewSQL database, but can also be used as a standalone storage system.

Pros

  • High performance and low latency due to its Rust implementation
  • Strong consistency and ACID compliance for distributed transactions
  • Horizontal scalability with automatic sharding
  • Supports various storage engines, including RocksDB

Cons

  • Steep learning curve for newcomers to distributed systems
  • Complex deployment and maintenance compared to traditional databases
  • Limited ecosystem and tooling compared to more established databases
  • Resource-intensive, requiring significant hardware resources for optimal performance

Getting Started

To get started with TiKV, follow these steps:

  1. Install TiKV using the official installation script:
curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
  1. Start a TiKV cluster using TiUP:
tiup playground
  1. Connect to the cluster using the TiKV client:
tiup client
  1. Use the TiKV client to interact with the database:
tikv> put mykey myvalue
tikv> get mykey

For more detailed instructions and advanced usage, refer to the official TiKV documentation.

Competitor Comparisons

37,777

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.

Pros of TiDB

  • Higher-level database system with SQL support
  • Designed for distributed HTAP (Hybrid Transactional/Analytical Processing) workloads
  • Provides a MySQL-compatible interface for easier adoption

Cons of TiDB

  • More complex setup and configuration compared to TiKV
  • Potentially higher resource requirements due to additional layers

Code Comparison

TiDB (SQL query execution):

impl<E: Engine> Executor for TableScanExecutor<E> {
    fn next(&mut self) -> Result<Option<Row>> {
        if let Some(row) = self.scanner.next()? {
            Ok(Some(row))
        } else {
            Ok(None)
        }
    }
}

TiKV (Key-Value operations):

impl Engine for RocksEngine {
    fn get(&self, key: &[u8]) -> Result<Option<Vec<u8>>> {
        let v = self.db.get(key)?;
        Ok(v.map(|v| v.to_vec()))
    }
}

TiDB is a distributed SQL database built on top of TiKV, offering a higher-level abstraction with SQL support. TiKV, on the other hand, is a distributed key-value store that serves as the storage layer for TiDB. While TiDB provides a more familiar SQL interface and HTAP capabilities, it comes with increased complexity and potential resource overhead compared to the simpler, lower-level TiKV.

30,408

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

Pros of CockroachDB

  • Built-in SQL layer with PostgreSQL compatibility
  • Stronger consistency guarantees (serializable isolation)
  • More mature ecosystem with official cloud offering (CockroachCloud)

Cons of CockroachDB

  • Higher resource consumption and complexity
  • Less flexible storage engine (RocksDB-based)
  • Steeper learning curve for operations and tuning

Code Comparison

TiKV (Rust):

pub struct Engine {
    db: RocksEngine,
    config: Config,
}

impl Engine {
    pub fn new(path: &str, config: Config) -> Result<Self> {
        let db = RocksEngine::new(path)?;
        Ok(Engine { db, config })
    }
}

CockroachDB (Go):

type Engine struct {
    db     *pebble.DB
    config Config
}

func NewEngine(path string, config Config) (*Engine, error) {
    db, err := pebble.Open(path, &pebble.Options{})
    if err != nil {
        return nil, err
    }
    return &Engine{db: db, config: config}, nil
}

Both projects use key-value storage engines (RocksDB for TiKV, Pebble for CockroachDB) but have different language implementations and slightly different APIs. CockroachDB provides a higher-level SQL interface on top of its storage engine, while TiKV focuses on providing a distributed key-value store that can be used as a building block for other database systems.

5,271

Apache HBase

Pros of HBase

  • Mature ecosystem with extensive documentation and community support
  • Strong integration with Hadoop ecosystem and MapReduce
  • Proven scalability for handling extremely large datasets

Cons of HBase

  • Higher latency compared to TiKV for read/write operations
  • More complex setup and maintenance requirements
  • Heavier resource consumption, especially for smaller deployments

Code Comparison

HBase (Java):

Put put = new Put(Bytes.toBytes("row1"));
put.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual1"), Bytes.toBytes("value1"));
table.put(put);

TiKV (Rust):

let client = TiKvClient::new(vec!["127.0.0.1:2379"]);
let key = b"key".to_vec();
let value = b"value".to_vec();
client.put(key, value).unwrap();

Summary

HBase is a mature, widely-adopted solution for large-scale data storage, particularly within the Hadoop ecosystem. It offers robust scalability and strong integration with other Big Data tools. However, it can be more complex to set up and maintain, and may have higher latency for certain operations.

TiKV, on the other hand, is designed for lower latency and easier deployment, making it potentially more suitable for smaller-scale or latency-sensitive applications. It also offers a more modern, modular architecture. However, it may not have the same level of ecosystem integration or community support as HBase.

The choice between the two depends on specific use cases, existing infrastructure, and performance requirements.

Apache Cassandra®

Pros of Cassandra

  • Highly scalable and distributed architecture, designed for large-scale deployments
  • Strong support for multi-datacenter replication and high availability
  • Mature project with a large community and extensive documentation

Cons of Cassandra

  • Complex setup and configuration process
  • Higher memory consumption compared to some alternatives
  • Less flexible query capabilities, primarily optimized for known access patterns

Code Comparison

Cassandra (CQL):

CREATE TABLE users (
  id UUID PRIMARY KEY,
  name TEXT,
  email TEXT
);

TiKV (TiDB SQL):

CREATE TABLE users (
  id UUID PRIMARY KEY,
  name VARCHAR(255),
  email VARCHAR(255)
);

Both systems use SQL-like syntax for schema definition, but TiKV (through TiDB) offers more SQL compatibility and advanced features like distributed transactions. Cassandra's data model is more denormalized and optimized for specific access patterns, while TiKV provides a more flexible approach to data modeling and querying.

TiKV offers a multi-model database system with support for both key-value and relational models, whereas Cassandra is primarily focused on wide-column store capabilities. This difference in design philosophy impacts how data is stored, accessed, and queried in each system.

48,227

Distributed reliable key-value store for the most critical data of a distributed system

Pros of etcd

  • Simpler architecture and easier to set up for small to medium-scale deployments
  • Native support for Kubernetes as its default key-value store
  • Lower resource consumption for smaller datasets

Cons of etcd

  • Limited scalability for very large datasets (typically <100GB)
  • Less flexible consistency model compared to TiKV's multi-version concurrency control
  • Fewer advanced features like distributed transactions and coprocessors

Code Comparison

etcd (Go):

cli, _ := clientv3.New(clientv3.Config{Endpoints: []string{"localhost:2379"}})
defer cli.Close()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
_, err := cli.Put(ctx, "key", "value")
cancel()

TiKV (Rust):

let client = TiKVClient::new(vec!["127.0.0.1:2379"]).await?;
let mut txn = client.begin().await?;
txn.put("key".to_owned(), "value".to_owned()).await?;
txn.commit().await?;

Both etcd and TiKV are distributed key-value stores, but they target different use cases. etcd is optimized for smaller datasets and is commonly used for service discovery and configuration management. TiKV, on the other hand, is designed for larger-scale deployments and offers more advanced features suitable for distributed database systems.

14,262

NoSQL data store using the Seastar framework, compatible with Apache Cassandra and Amazon DynamoDB

Pros of ScyllaDB

  • Higher performance and lower latency due to its C++ implementation and shared-nothing architecture
  • Better resource utilization and scalability, capable of handling larger datasets on fewer nodes
  • Compatibility with Apache Cassandra, making migration easier for existing Cassandra users

Cons of ScyllaDB

  • Less flexible in terms of data models and query patterns compared to TiKV's key-value store approach
  • Smaller community and ecosystem compared to TiKV, which is part of the larger CNCF landscape
  • More complex setup and maintenance, especially for smaller deployments

Code Comparison

ScyllaDB (CQL):

CREATE TABLE users (
  user_id UUID PRIMARY KEY,
  name TEXT,
  email TEXT
);

TiKV (Rust client):

let key = b"user:1001".to_vec();
let value = json!({
    "name": "John Doe",
    "email": "john@example.com"
}).to_string().into_bytes();
client.put(key, value).await?;

Both databases offer different approaches to data storage and retrieval. ScyllaDB uses a table-based model with CQL, while TiKV provides a more flexible key-value store interface. The choice between them depends on specific use cases and requirements.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

tikv_logo

Website | Documentation | Community Chat

Build Status Coverage Status CII Best Practices

TiKV is an open-source, distributed, and transactional key-value database. Unlike other traditional NoSQL systems, TiKV not only provides classical key-value APIs, but also transactional APIs with ACID compliance. Built in Rust and powered by Raft, TiKV was originally created by PingCAP to complement TiDB, a distributed HTAP database compatible with the MySQL protocol.

The design of TiKV ('Ti' stands for titanium) is inspired by some great distributed systems from Google, such as BigTable, Spanner, and Percolator, and some of the latest achievements in academia in recent years, such as the Raft consensus algorithm.

If you're interested in contributing to TiKV, or want to build it from source, see CONTRIBUTING.md.

cncf_logo cncf_logo

TiKV is a graduated project of the Cloud Native Computing Foundation (CNCF). If you are an organization that wants to help shape the evolution of technologies that are container-packaged, dynamically-scheduled and microservices-oriented, consider joining the CNCF. For details about who's involved and how TiKV plays a role, read the CNCF announcement.


With the implementation of the Raft consensus algorithm in Rust and consensus state stored in RocksDB, TiKV guarantees data consistency. Placement Driver (PD), which is introduced to implement auto-sharding, enables automatic data migration. The transaction model is similar to Google's Percolator with some performance improvements. TiKV also provides snapshot isolation (SI), snapshot isolation with lock (SQL: SELECT ... FOR UPDATE), and externally consistent reads and writes in distributed transactions.

TiKV has the following key features:

  • Geo-Replication

    TiKV uses Raft and the Placement Driver to support Geo-Replication.

  • Horizontal scalability

    With PD and carefully designed Raft groups, TiKV excels in horizontal scalability and can easily scale to 100+ TBs of data.

  • Consistent distributed transactions

    Similar to Google's Spanner, TiKV supports externally-consistent distributed transactions.

  • Coprocessor support

    Similar to HBase, TiKV implements a coprocessor framework to support distributed computing.

  • Cooperates with TiDB

    Thanks to the internal optimization, TiKV and TiDB can work together to be a compelling database solution with high horizontal scalability, externally-consistent transactions, support for RDBMS, and NoSQL design patterns.

Governance

See Governance.

Documentation

For instructions on deployment, configuration, and maintenance of TiKV,see TiKV documentation on our website. For more details on concepts and designs behind TiKV, see Deep Dive TiKV.

Note:

We have migrated our documentation from the TiKV's wiki page to the official website. The original Wiki page is discontinued. If you have any suggestions or issues regarding documentation, offer your feedback here.

TiKV adopters

You can view the list of TiKV Adopters.

TiKV software stack

The TiKV software stack

  • Placement Driver: PD is the cluster manager of TiKV, which periodically checks replication constraints to balance load and data automatically.
  • Store: There is a RocksDB within each Store and it stores data into the local disk.
  • Region: Region is the basic unit of Key-Value data movement. Each Region is replicated to multiple Nodes. These multiple replicas form a Raft group.
  • Node: A physical node in the cluster. Within each node, there are one or more Stores. Within each Store, there are many Regions.

When a node starts, the metadata of the Node, Store and Region are recorded into PD. The status of each Region and Store is reported to PD regularly.

Quick start

Deploy a playground with TiUP

The most quickest to try out TiKV with TiDB is using TiUP, a component manager for TiDB.

You can see this page for a step by step tutorial.

Deploy a playground with binary

TiKV is able to run separately with PD, which is the minimal deployment required.

  1. Download and extract binaries.
$ export TIKV_VERSION=v7.5.0
$ export GOOS=darwin  # only {darwin, linux} are supported
$ export GOARCH=amd64 # only {amd64, arm64} are supported
$ curl -O  https://tiup-mirrors.pingcap.com/tikv-$TIKV_VERSION-$GOOS-$GOARCH.tar.gz
$ curl -O  https://tiup-mirrors.pingcap.com/pd-$TIKV_VERSION-$GOOS-$GOARCH.tar.gz
$ tar -xzf tikv-$TIKV_VERSION-$GOOS-$GOARCH.tar.gz
$ tar -xzf pd-$TIKV_VERSION-$GOOS-$GOARCH.tar.gz
  1. Start PD instance.
$ ./pd-server --name=pd --data-dir=/tmp/pd/data --client-urls="http://127.0.0.1:2379" --peer-urls="http://127.0.0.1:2380" --initial-cluster="pd=http://127.0.0.1:2380" --log-file=/tmp/pd/log/pd.log
  1. Start TiKV instance.
$ ./tikv-server --pd-endpoints="127.0.0.1:2379" --addr="127.0.0.1:20160" --data-dir=/tmp/tikv/data --log-file=/tmp/tikv/log/tikv.log
  1. Install TiKV Client(Python) and verify the deployment, required Python 3.5+.
$ pip3 install -i https://test.pypi.org/simple/ tikv-client
from tikv_client import RawClient

client = RawClient.connect("127.0.0.1:2379")

client.put(b'foo', b'bar')
print(client.get(b'foo')) # b'bar'

client.put(b'foo', b'baz')
print(client.get(b'foo')) # b'baz'

Deploy a cluster with TiUP

You can see this manual of production-like cluster deployment presented by @c4pt0r.

Build from source

See CONTRIBUTING.md.

Client drivers

If you want to try the Go client, see Go Client.

Security

Security audit

A third-party security auditing was performed by Cure53. See the full report here.

Reporting Security Vulnerabilities

To report a security vulnerability, please send an email to TiKV-security group.

See Security for the process and policy followed by the TiKV project.

Communication

Communication within the TiKV community abides by TiKV Code of Conduct. Here is an excerpt:

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.

Social Media

Slack

Join the TiKV community on Slack - Sign up and join channels on TiKV topics that interest you.

License

TiKV is under the Apache 2.0 license. See the LICENSE file for details.

Acknowledgments

  • Thanks etcd for providing some great open source tools.
  • Thanks RocksDB for their powerful storage engines.
  • Thanks rust-clippy. We do love the great project.