Convert Figma logo to code with AI

twitter-archive logosnowflake

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees.

7,646
1,135
7,646
4

Top Related Projects

A distributed unique ID generator inspired by Twitter's Snowflake

4,432

Universally Unique Lexicographically Sortable Identifier (ULID) in Go

3,421

Collision-resistant ids optimized for horizontal scaling and performance.

Quick Overview

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees. It was originally created by Twitter to provide unique IDs for tweets and other entities in their system. Snowflake is designed to generate roughly sortable unique 64-bit IDs in a distributed environment.

Pros

  • High scalability: Can generate up to 4096 unique IDs per millisecond per process
  • Roughly time sortable: IDs can be used to determine the order of events
  • Distributed: Can be run on multiple machines without coordination
  • Customizable: Allows for custom epoch and machine ID configuration

Cons

  • Requires synchronized clocks: Relies on system clocks being relatively in sync
  • Potential for duplicate IDs: If system clock moves backwards, duplicates can occur
  • Limited lifespan: Will eventually run out of IDs (after ~69 years with default settings)
  • Not cryptographically secure: IDs are predictable and should not be used for security purposes

Code Examples

  1. Generating a new Snowflake ID:
IdWorker idWorker = new IdWorker(0, 0);
long id = idWorker.nextId();
System.out.println("Generated ID: " + id);
  1. Extracting timestamp from a Snowflake ID:
long id = 1234567890123456789L;
long timestamp = (id >> 22) + IdWorker.twepoch;
System.out.println("Timestamp: " + new Date(timestamp));
  1. Creating a custom IdWorker with a different epoch:
long customEpoch = 1609459200000L; // January 1, 2021
IdWorker customIdWorker = new IdWorker(0, 0, customEpoch);
long customId = customIdWorker.nextId();
System.out.println("Custom ID: " + customId);

Getting Started

To use Snowflake in your Java project:

  1. Clone the repository:

    git clone https://github.com/twitter-archive/snowflake.git
    
  2. Build the project:

    cd snowflake
    mvn clean install
    
  3. Add the dependency to your project's pom.xml:

    <dependency>
      <groupId>com.twitter</groupId>
      <artifactId>snowflake</artifactId>
      <version>1.0.0</version>
    </dependency>
    
  4. Use the IdWorker class in your code:

    import com.twitter.snowflake.IdWorker;
    
    IdWorker idWorker = new IdWorker(0, 0);
    long id = idWorker.nextId();
    

Competitor Comparisons

A distributed unique ID generator inspired by Twitter's Snowflake

Pros of Sonyflake

  • Written in Go, offering better performance and concurrency
  • Supports custom epoch times, allowing for more flexible timestamp ranges
  • Provides a simpler API with fewer dependencies

Cons of Sonyflake

  • Less widely adopted compared to Snowflake
  • Lacks some advanced features like worker ID assignment

Code Comparison

Snowflake (Ruby):

require 'snowflake-id'

snowflake = Snowflake::Id.new(worker_id: 1, datacenter_id: 1)
id = snowflake.next_id

Sonyflake (Go):

import "github.com/sony/sonyflake"

flake := sonyflake.NewSonyflake(sonyflake.Settings{})
id, err := flake.NextID()

Both Snowflake and Sonyflake are distributed unique ID generators, but they differ in implementation and features. Snowflake, originally developed by Twitter, is widely adopted and has implementations in various languages. Sonyflake, developed by Sony, offers a Go-based alternative with some unique features.

Sonyflake provides better performance due to Go's efficiency and supports custom epoch times. However, it has a smaller community and fewer advanced features compared to Snowflake. The choice between the two depends on specific project requirements, language preferences, and desired features.

4,432

Universally Unique Lexicographically Sortable Identifier (ULID) in Go

Pros of ULID

  • Language-agnostic implementation with support for multiple programming languages
  • Sortable by timestamp, allowing for easy chronological ordering
  • Compact representation (26 characters) compared to Snowflake's 64-bit integer

Cons of ULID

  • Lacks built-in distributed system support, unlike Snowflake's datacenter-aware design
  • Potential for collisions in high-volume scenarios due to limited entropy

Code Comparison

Snowflake (Java):

long id = snowflake.nextId();

ULID (Go):

id := ulid.New()

Key Differences

  • Snowflake focuses on distributed systems and high-scale environments
  • ULID prioritizes simplicity and cross-language compatibility
  • Snowflake uses a 64-bit integer, while ULID uses a 26-character string
  • Snowflake includes machine ID in its structure, ULID does not

Use Cases

Snowflake:

  • Large-scale distributed systems
  • High-throughput environments requiring unique IDs

ULID:

  • Cross-platform applications
  • Systems prioritizing human-readable and sortable IDs

Both solutions offer unique identifier generation, but their designs cater to different priorities and use cases.

3,421

Collision-resistant ids optimized for horizontal scaling and performance.

Pros of cuid

  • Simpler implementation, easier to understand and maintain
  • More portable, can be used in various environments (browser, server, mobile)
  • Generates shorter IDs, which can be beneficial for storage and transmission

Cons of cuid

  • Less scalable for extremely high-volume distributed systems
  • May have a slightly higher chance of collisions in certain scenarios
  • Lacks some advanced features like worker ID and datacenter ID

Code Comparison

Snowflake (Java):

public synchronized long nextId() {
    long timestamp = timeGen();
    if (timestamp < lastTimestamp) {
        throw new RuntimeException("Clock moved backwards.");
    }
    if (lastTimestamp == timestamp) {
        sequence = (sequence + 1) & sequenceMask;
        if (sequence == 0) {
            timestamp = tilNextMillis(lastTimestamp);
        }
    } else {
        sequence = 0L;
    }
    lastTimestamp = timestamp;
    return ((timestamp - twepoch) << timestampLeftShift) |
           (datacenterId << datacenterIdShift) |
           (workerId << workerIdShift) |
           sequence;
}

cuid (JavaScript):

function cuid() {
  return 'c' + (Date.now()).toString(36) +
         counter().toString(36).slice(-4) +
         fingerprint() +
         random();
}

The Snowflake implementation is more complex and includes features for distributed systems, while cuid offers a simpler approach that's easier to implement across different platforms.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

We have retired the initial release of Snowflake and working on open sourcing the next version based on Twitter-server, in a form that can run anywhere without requiring Twitter's own infrastructure services.

The initial version, released in 2010, was based on Apache Thrift and it predated Finagle, our building block for RPC services at Twitter. The Snowflake we're using internally is a full rewrite and heavily relies on existing infrastructure at Twitter to run. We cannot commit to a date but we're doing our best to add necessary features to make Snowflake fit for many environments outside of Twitter.

Source code is still in the repository and is reachable from snowflake-2010 tag.

We won't be accepting pull requests or responding to issues for the retired release.