Convert Figma logo to code with AI

sony logosonyflake

A distributed unique ID generator inspired by Twitter's Snowflake

3,862
304
3,862
9

Top Related Projects

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees.

3,421

Collision-resistant ids optimized for horizontal scaling and performance.

9,560

The canonical spec for ulid

4,432

Universally Unique Lexicographically Sortable Identifier (ULID) in Go

3,879

xid is a globally unique id generator thought for the web

Quick Overview

Sonyflake is a distributed unique ID generator inspired by Twitter's Snowflake. It generates 63-bit unique IDs composed of a timestamp, machine ID, and sequence number. Sonyflake is designed to work in distributed systems and can generate IDs for 174 years without exhaustion.

Pros

  • High performance and low latency ID generation
  • Supports distributed systems with machine ID component
  • IDs are sortable by time of creation
  • Customizable settings for timestamp, machine ID, and sequence bits

Cons

  • Requires careful clock synchronization across machines
  • Limited to 63-bit IDs, which may not be suitable for all use cases
  • Potential for ID collisions if not configured properly
  • Relatively small community and limited ecosystem compared to some alternatives

Code Examples

  1. Basic ID generation:
import "github.com/sony/sonyflake"

sf := sonyflake.NewSonyflake(sonyflake.Settings{})
id, err := sf.NextID()
if err != nil {
    // Handle error
}
fmt.Printf("Generated ID: %d\n", id)
  1. Custom settings:
import "github.com/sony/sonyflake"

settings := sonyflake.Settings{
    StartTime: time.Date(2023, 1, 1, 0, 0, 0, 0, time.UTC),
    MachineID: func() (uint16, error) {
        return 1, nil // Custom machine ID
    },
}
sf := sonyflake.NewSonyflake(settings)
  1. Extracting components from an ID:
import "github.com/sony/sonyflake"

id := uint64(123456789)
parts := sonyflake.Decompose(id)
fmt.Printf("Timestamp: %d\n", parts["time"])
fmt.Printf("Machine ID: %d\n", parts["machine-id"])
fmt.Printf("Sequence: %d\n", parts["sequence"])

Getting Started

To use Sonyflake in your Go project:

  1. Install the package:

    go get github.com/sony/sonyflake
    
  2. Import and use in your code:

    import "github.com/sony/sonyflake"
    
    sf := sonyflake.NewSonyflake(sonyflake.Settings{})
    id, err := sf.NextID()
    if err != nil {
        // Handle error
    }
    fmt.Printf("Generated ID: %d\n", id)
    

Ensure proper error handling and consider customizing settings for your specific use case.

Competitor Comparisons

Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees.

Pros of Snowflake

  • Original implementation, widely recognized and adopted
  • Designed for high-scale distributed systems
  • Includes machine ID in the generated ID, useful for data center identification

Cons of Snowflake

  • Written in Scala, which may not be as widely used as Go
  • Less actively maintained (archived repository)
  • More complex setup and configuration compared to Sonyflake

Code Comparison

Snowflake (Scala):

def nextId(): Long = {
  var timestamp = timeGen()
  var id = ((timestamp - twepoch) << timestampLeftShift) |
    (datacenterId << datacenterIdShift) |
    (workerId << workerIdShift) |
    sequence
  id
}

Sonyflake (Go):

func (sf *Sonyflake) NextID() (uint64, error) {
	const maskSequence = uint16(1<<BitLenSequence - 1)
	elapsedTime := uint64(sf.elapsedTime(sf.startTime))
	sf.mutex.Lock()
	if sf.elapsedTime > elapsedTime {
		sf.mutex.Unlock()
		return 0, errors.New("time is backwards")
	}
	if sf.elapsedTime == elapsedTime {
		sf.sequence = (sf.sequence + 1) & maskSequence
		if sf.sequence == 0 {
			sf.mutex.Unlock()
			return 0, errors.New("over the sequence number")
		}
	} else {
		sf.sequence = 0
	}
	id := elapsedTime<<(BitLenSequence+BitLenMachineID) |
		uint64(sf.machineID)<<BitLenSequence |
		uint64(sf.sequence)
	sf.elapsedTime = elapsedTime
	sf.mutex.Unlock()
	return id, nil
}

Both implementations generate unique IDs, but Sonyflake's Go code is more concise and easier to integrate into modern microservices architectures.

3,421

Collision-resistant ids optimized for horizontal scaling and performance.

Pros of cuid

  • Simpler implementation, easier to understand and use
  • Supports multiple programming languages and environments
  • Generates shorter IDs (20-25 characters) compared to Sonyflake's 64-bit integers

Cons of cuid

  • Less efficient for high-volume ID generation
  • May have a higher chance of collisions in extremely large datasets
  • Lacks built-in timestamp extraction functionality

Code Comparison

cuid:

import { createId } from '@paralleldrive/cuid2';

const id = createId();
console.log(id); // Example output: ckwqq6vnz000001la036pezgh

Sonyflake:

import "github.com/sony/sonyflake"

sf := sonyflake.NewSonyflake(sonyflake.Settings{})
id, err := sf.NextID()
if err != nil {
    panic(err)
}
fmt.Printf("%d\n", id) // Example output: 120364834234830848

Both cuid and Sonyflake provide unique ID generation solutions, but they cater to different use cases. cuid offers simplicity and broad language support, making it suitable for various applications. Sonyflake, on the other hand, is optimized for high-volume ID generation and includes timestamp extraction capabilities, making it more appropriate for large-scale distributed systems.

9,560

The canonical spec for ulid

Pros of ULID

  • Lexicographically sortable, allowing for efficient database indexing
  • Includes a timestamp component for easy time-based sorting
  • Language-agnostic specification, enabling implementations in various programming languages

Cons of ULID

  • Slightly longer identifier (26 characters) compared to Sonyflake's 16-character IDs
  • Requires more complex implementation due to the combination of timestamp and randomness

Code Comparison

ULID (JavaScript implementation):

function ulid() {
  return timestamp() + randomness();
}

Sonyflake (Go implementation):

func (sf *Sonyflake) NextID() (uint64, error) {
    return sf.nextID()
}

The ULID code snippet demonstrates the basic structure of generating a ULID by combining a timestamp and random component. The Sonyflake code shows the method for generating the next ID using the Sonyflake struct.

Both solutions provide unique identifier generation, but ULID offers more flexibility and sortability at the cost of slightly longer IDs and more complex implementation. Sonyflake, on the other hand, provides a simpler, more compact solution that may be sufficient for many use cases.

4,432

Universally Unique Lexicographically Sortable Identifier (ULID) in Go

Pros of ULID

  • Lexicographically sortable, allowing for easy chronological ordering
  • Includes millisecond precision timestamp, providing more granular time information
  • Supports multiple programming languages and has various implementations

Cons of ULID

  • Slightly longer identifier (26 characters) compared to Sonyflake's 16 characters
  • May have higher memory usage due to larger size
  • Potential for timestamp collisions in high-volume scenarios

Code Comparison

ULID:

id := ulid.MustNew(ulid.Timestamp(time.Now()), entropy)

Sonyflake:

flake := sonyflake.NewSonyflake(sonyflake.Settings{})
id, err := flake.NextID()

Both ULID and Sonyflake are distributed unique ID generation systems, but they differ in their approach and features. ULID offers better sortability and more precise timestamps, while Sonyflake provides shorter IDs and potentially better performance in high-volume scenarios. The choice between them depends on specific project requirements, such as the need for chronological ordering, ID length constraints, or multi-language support.

3,879

xid is a globally unique id generator thought for the web

Pros of xid

  • Smaller ID size (20 bytes) compared to Sonyflake's 64-bit IDs
  • Built-in support for multiple programming languages (Go, Rust, Python, etc.)
  • Includes machine ID and process ID in the generated IDs

Cons of xid

  • Less customizable than Sonyflake (e.g., can't adjust timestamp precision)
  • May have slightly higher collision probability due to smaller ID space
  • Not specifically designed for distributed systems like Sonyflake

Code Comparison

xid:

id := xid.New()
fmt.Printf("xid: %s\n", id.String())

Sonyflake:

flake := sonyflake.NewSonyflake(sonyflake.Settings{})
id, err := flake.NextID()
fmt.Printf("Sonyflake ID: %d\n", id)

Both xid and Sonyflake provide unique ID generation, but they differ in implementation and use cases. xid offers a more compact ID format and multi-language support, making it suitable for various applications. Sonyflake, designed specifically for distributed systems, provides more customization options and a larger ID space, potentially reducing collision risks in high-volume scenarios.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Sonyflake

GoDoc Go Report Card

Sonyflake is a distributed unique ID generator inspired by Twitter's Snowflake.

Sonyflake focuses on lifetime and performance on many host/core environment. So it has a different bit assignment from Snowflake. A Sonyflake ID is composed of

39 bits for time in units of 10 msec
 8 bits for a sequence number
16 bits for a machine id

As a result, Sonyflake has the following advantages and disadvantages:

  • The lifetime (174 years) is longer than that of Snowflake (69 years)
  • It can work in more distributed machines (2^16) than Snowflake (2^10)
  • It can generate 2^8 IDs per 10 msec at most in a single machine/thread (slower than Snowflake)

However, if you want more generation rate in a single host, you can easily run multiple Sonyflake ID generators concurrently using goroutines.

Installation

go get github.com/sony/sonyflake

Usage

The function New creates a new Sonyflake instance.

func New(st Settings) (*Sonyflake, error)

You can configure Sonyflake by the struct Settings:

type Settings struct {
	StartTime      time.Time
	MachineID      func() (uint16, error)
	CheckMachineID func(uint16) bool
}
  • StartTime is the time since which the Sonyflake time is defined as the elapsed time. If StartTime is 0, the start time of the Sonyflake is set to "2014-09-01 00:00:00 +0000 UTC". If StartTime is ahead of the current time, Sonyflake is not created.

  • MachineID returns the unique ID of the Sonyflake instance. If MachineID returns an error, Sonyflake is not created. If MachineID is nil, default MachineID is used. Default MachineID returns the lower 16 bits of the private IP address.

  • CheckMachineID validates the uniqueness of the machine ID. If CheckMachineID returns false, Sonyflake is not created. If CheckMachineID is nil, no validation is done.

In order to get a new unique ID, you just have to call the method NextID.

func (sf *Sonyflake) NextID() (uint64, error)

NextID can continue to generate IDs for about 174 years from StartTime. But after the Sonyflake time is over the limit, NextID returns an error.

Note: Sonyflake currently does not use the most significant bit of IDs, so you can convert Sonyflake IDs from uint64 to int64 safely.

AWS VPC and Docker

The awsutil package provides the function AmazonEC2MachineID that returns the lower 16-bit private IP address of the Amazon EC2 instance. It also works correctly on Docker by retrieving instance metadata.

AWS VPC is assigned a single CIDR with a netmask between /28 and /16. So if each EC2 instance has a unique private IP address in AWS VPC, the lower 16 bits of the address is also unique. In this common case, you can use AmazonEC2MachineID as Settings.MachineID.

See example that runs Sonyflake on AWS Elastic Beanstalk.

License

The MIT License (MIT)

See LICENSE for details.