haxmap

Fastest and most memory efficient golang concurrent hashmap

1,001

View on GitHub

Top Related Projects

hashmap

1,859

A Golang lock-free thread-safe HashMap optimized for fastest read access.

concurrent-map

4,454

a thread-safe concurrent map for go

gods

16,988

GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more

golang-set

4,538

A simple, battle-tested and generic set type for the Go language. Trusted by Docker, 1Password, Ethereum and Hashicorp.

xxhash

1,773

A Go implementation of the 64-bit xxHash algorithm (XXH64)

Quick Overview

HaxMap is a high-performance, thread-safe concurrent hashmap for Go. It offers better performance than the standard sync.Map and is designed to handle heavy concurrent read/write operations efficiently.

Pros

Significantly faster than sync.Map for concurrent read/write operations
Thread-safe implementation
Zero dependencies, pure Go implementation
Supports generics for type safety

Cons

Relatively new project, may not be as battle-tested as sync.Map
Limited documentation and examples compared to standard library
May have a steeper learning curve for developers used to sync.Map

Code Examples

Creating and using a HaxMap:

import "github.com/alphadose/haxmap"

// Create a new HaxMap with string keys and int values
m := haxmap.New[string, int]()

// Insert a key-value pair
m.Set("foo", 42)

// Get a value
val, ok := m.Get("foo")
if ok {
    fmt.Println(val) // Output: 42
}

Concurrent operations:

m := haxmap.New[int, string]()

// Concurrent writes
for i := 0; i < 1000; i++ {
    go func(i int) {
        m.Set(i, fmt.Sprintf("value-%d", i))
    }(i)
}

// Concurrent reads
for i := 0; i < 1000; i++ {
    go func(i int) {
        if val, ok := m.Get(i); ok {
            fmt.Printf("Key %d: %s\n", i, val)
        }
    }(i)
}

Using ForEach method:

m := haxmap.New[string, int]()
m.Set("a", 1)
m.Set("b", 2)
m.Set("c", 3)

m.ForEach(func(key string, value int) bool {
    fmt.Printf("%s: %d\n", key, value)
    return true // continue iteration
})

Getting Started

To use HaxMap in your Go project, follow these steps:

Install the package:
```
go get github.com/alphadose/haxmap
```
Import the package in your Go code:
```
import "github.com/alphadose/haxmap"
```

Create and use a HaxMap:

m := haxmap.New[string, int]()
m.Set("key", 42)
value, exists := m.Get("key")

Competitor Comparisons

hashmap

1,859

A Golang lock-free thread-safe HashMap optimized for fastest read access.

Pros of hashmap

More comprehensive test suite with higher code coverage
Supports custom hash functions for greater flexibility
Includes benchmarks for performance evaluation

Cons of hashmap

Slightly more complex API with additional methods
May have higher memory usage due to more features

Code Comparison

haxmap:

type Map[K comparable, V any] struct {
    shards [256]unsafe.Pointer
    hasher func(K) uint64
}

hashmap:

type Map[Key comparable, Value any] struct {
    count     int64
    mapData   mapData[Key, Value]
    hasher    func(Key) uint64
    growThreshold int
}

Key Differences

haxmap uses a fixed number of shards (256), while hashmap has a more flexible structure
hashmap includes a count field for tracking the number of elements
haxmap's structure is simpler, potentially leading to better performance in some cases
hashmap offers more customization options, such as custom growth thresholds

Both libraries provide concurrent hash map implementations in Go, but they differ in their approach to balancing simplicity and features. haxmap focuses on high performance with a simpler structure, while hashmap offers more flexibility and customization options at the cost of slightly increased complexity.

concurrent-map

4,454

a thread-safe concurrent map for go

Pros of concurrent-map

More established project with a longer history and wider adoption
Provides additional utility functions like Items() and IterBuffered()
Offers a simpler API with fewer methods, potentially easier to use for basic scenarios

Cons of concurrent-map

Generally slower performance compared to haxmap, especially for read operations
Lacks advanced features like expiration and custom hash functions
Does not support generics, requiring type assertions when retrieving values

Code Comparison

concurrent-map:

m := cmap.New()
m.Set("key", "value")
value, ok := m.Get("key")

haxmap:

m := haxmap.New[string, string]()
m.Set("key", "value")
value, ok := m.Get("key")

The basic usage is similar, but haxmap supports generics, eliminating the need for type assertions when retrieving values. haxmap also offers more advanced features and better performance, particularly for read-heavy workloads. However, concurrent-map may be sufficient for simpler use cases and has a longer track record in production environments.

gods

16,988

GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more

Pros of gods

Offers a wider variety of data structures (lists, stacks, maps, trees, etc.)
More comprehensive documentation and examples
Higher number of stars and contributors, indicating broader community adoption

Cons of gods

Generally slower performance compared to haxmap
Less focused on specific use cases, potentially leading to suboptimal implementations for certain scenarios
Larger codebase, which may increase complexity for simple use cases

Code Comparison

haxmap:

m := haxmap.New[string, int]()
m.Set("key", 10)
value, _ := m.Get("key")

gods:

m := hashmap.New()
m.Put("key", 10)
value, _ := m.Get("key")

Both libraries provide similar basic functionality for hash maps, but haxmap is more focused on performance and type safety, while gods offers a broader range of data structures and operations.

golang-set

4,538

A simple, battle-tested and generic set type for the Go language. Trusted by Docker, 1Password, Ethereum and Hashicorp.

Pros of golang-set

Implements a classic set data structure with well-defined set operations
Provides thread-safe set implementations for concurrent use
Supports various data types, including custom types that implement Hashable

Cons of golang-set

Generally slower performance compared to haxmap
Higher memory usage due to additional set-specific functionality
Less flexible for use cases beyond set operations

Code Comparison

golang-set:

set := mapset.NewSet()
set.Add("apple")
set.Add("banana")
fmt.Println(set.Contains("apple"))

haxmap:

m := haxmap.New[string, struct{}]()
m.Set("apple", struct{}{})
m.Set("banana", struct{}{})
fmt.Println(m.Get("apple"))

Key Differences

golang-set focuses on set operations, while haxmap is a general-purpose concurrent map
haxmap offers better performance and lower memory usage
golang-set provides more set-specific functionality out of the box
haxmap requires manual implementation of set-like behavior using empty structs as values

Use Cases

golang-set is ideal for:

Set-specific operations and algorithms
Thread-safe set implementations
Projects requiring a traditional set data structure

haxmap is better suited for:

High-performance concurrent map operations
Memory-efficient key-value storage
Custom implementations of set-like behavior with additional flexibility

xxhash

1,773

A Go implementation of the 64-bit xxHash algorithm (XXH64)

Pros of xxhash

Widely adopted and battle-tested implementation of the xxHash algorithm
Offers both 32-bit and 64-bit hash functions
Provides a simple and straightforward API

Cons of xxhash

Limited to xxHash algorithm only
Does not offer concurrent access or thread-safety out of the box

Code Comparison

xxhash:

hash := xxhash.New()
hash.Write([]byte("hello world"))
sum := hash.Sum64()

haxmap:

hmap := haxmap.New[string, int]()
hmap.Set("key", 42)
value, ok := hmap.Get("key")

Key Differences

xxhash focuses solely on providing hash functions, while haxmap is a concurrent hash map implementation
haxmap offers built-in concurrency support, whereas xxhash requires additional synchronization for concurrent use
xxhash is more suitable for general-purpose hashing needs, while haxmap is designed for high-performance concurrent map operations

Use Cases

Choose xxhash when you need a fast and reliable hash function for various applications
Opt for haxmap when you require a concurrent hash map with high performance in multi-threaded environments

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

HaxMap

A lightning fast concurrent hashmap

The hashing algorithm used was xxHash and the hashmap's buckets were implemented using Harris lock-free list

Installation

You need Golang 1.18.x or above

$ go get github.com/alphadose/haxmap

Usage

package main

import (
	"fmt"

	"github.com/alphadose/haxmap"
)

func main() {
	// initialize map with key type `int` and value type `string`
	mep := haxmap.New[int, string]()

	// set a value (overwrites existing value if present)
	mep.Set(1, "one")

	// get the value and print it
	val, ok := mep.Get(1)
	if ok {
		println(val)
	}

	mep.Set(2, "two")
	mep.Set(3, "three")
	mep.Set(4, "four")

	// ForEach loop to iterate over all key-value pairs and execute the given lambda
	mep.ForEach(func(key int, value string) bool {
		fmt.Printf("Key -> %d | Value -> %s\n", key, value)
		return true // return `true` to continue iteration and `false` to break iteration
	})

	mep.Del(1) // delete a value
	mep.Del(0) // delete is safe even if a key doesn't exists

	// bulk deletion is supported too in the same API call
	// has better performance than deleting keys one by one
	mep.Del(2, 3, 4)

	if mep.Len() == 0 {
		println("cleanup complete")
	}
}

Benchmarks

Benchmarks were performed against golang sync.Map and the latest cornelk-hashmap

All results were computed from benchstat of 20 runs (code available here)

Concurrent Reads Only

name                         time/op
HaxMapReadsOnly-8            6.94Âµs Â± 4%
GoSyncMapReadsOnly-8         21.5Âµs Â± 3%
CornelkMapReadsOnly-8        8.39Âµs Â± 8%

Concurrent Reads with Writes

name                         time/op
HaxMapReadsWithWrites-8      8.23Âµs Â± 3%
GoSyncMapReadsWithWrites-8   25.0Âµs Â± 2%
CornelkMapReadsWithWrites-8  8.83Âµs Â±20%

name                         alloc/op
HaxMapReadsWithWrites-8      1.25kB Â± 5%
GoSyncMapReadsWithWrites-8   6.20kB Â± 7%
CornelkMapReadsWithWrites-8  1.53kB Â± 9%

name                         allocs/op
HaxMapReadsWithWrites-8         156 Â± 5%
GoSyncMapReadsWithWrites-8      574 Â± 7%
CornelkMapReadsWithWrites-8     191 Â± 9%

From the above results it is evident that haxmap takes the least time, memory and allocations in all cases making it the best golang concurrent hashmap in this period of time

Tips

HaxMap by default uses xxHash algorithm, but you can override this and plug-in your own custom hash function. Beneath lies an example for the same.

package main

import (
	"github.com/alphadose/haxmap"
)

// your custom hash function
// the hash function signature must adhere to `func(keyType) uintptr`
func customStringHasher(s string) uintptr {
	return uintptr(len(s))
}

func main() {
	m := haxmap.New[string, string]() // initialize a string-string map
	m.SetHasher(customStringHasher) // this overrides the default xxHash algorithm

	m.Set("one", "1")
	val, ok := m.Get("one")
	if ok {
		println(val)
	}
}

You can pre-allocate the size of the map which will improve performance in some cases.

package main

import (
	"github.com/alphadose/haxmap"
)

func main() {
	const initialSize = 1 << 10

	// pre-allocating the size of the map will prevent all grow operations
	// until that limit is hit thereby improving performance
	m := haxmap.New[int, string](initialSize)

	m.Set(1, "1")
	val, ok := m.Get(1)
	if ok {
		println(val)
	}
}

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot