bigcache

Efficient cache for gigabytes of data written in Go.

7,459

593

7,459

View on GitHub

Top Related Projects

go-cache

8,063

An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

gcache

2,578

An in-memory cache library for golang. It supports multiple eviction policies: LRU, LFU, ARC

freecache

5,056

A cache library for Go with zero GC overhead.

ristretto

5,532

A high performance memory-bound Go cache

Quick Overview

BigCache is a high-performance, in-memory cache library for Go. It is designed to be a fast and efficient cache solution for applications that require low-latency access to frequently used data. BigCache uses a custom hash table implementation and a specialized eviction strategy to provide high throughput and low memory usage.

Pros

High Performance: BigCache is designed to be extremely fast, with low latency and high throughput.
Low Memory Usage: The custom hash table implementation and eviction strategy allow BigCache to use memory efficiently, making it suitable for use in memory-constrained environments.
Customizable: BigCache provides a range of configuration options, allowing users to fine-tune its behavior to suit their specific needs.
Actively Maintained: The BigCache project is actively maintained by the Allegro team, with regular updates and bug fixes.

Cons

Limited Data Types: BigCache is designed to store and retrieve byte slices, which may not be suitable for all use cases.
Complexity: The custom hash table implementation and eviction strategy used by BigCache can be complex, which may make it more difficult to understand and debug.
Lack of Persistence: BigCache is an in-memory cache, which means that data stored in the cache will be lost if the application is restarted or the cache is cleared.
Limited Concurrency: While BigCache is designed to be thread-safe, it may not scale as well as other caching solutions in highly concurrent environments.

Code Examples

Here are a few examples of how to use BigCache in Go:

Initializing a BigCache instance:

cache, err := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
if err != nil {
    // handle error
}

Storing and retrieving data:

err := cache.Set("my-key", []byte("my-value"))
if err != nil {
    // handle error
}

value, err := cache.Get("my-key")
if err != nil {
    // handle error
}

Deleting data:

err := cache.Delete("my-key")
if err != nil {
    // handle error
}

Iterating over cache entries:

cache.Iterator(func(key, value []byte) bool {
    // process key and value
    return true // continue iteration
})

Getting Started

To get started with BigCache, you can follow these steps:

Install the BigCache package:

go get github.com/allegro/bigcache

Import the BigCache package in your Go code:

import "github.com/allegro/bigcache"

Create a new BigCache instance with the desired configuration:

config := bigcache.DefaultConfig(10 * time.Minute)
cache, err := bigcache.NewBigCache(config)
if err != nil {
    // handle error
}

Use the BigCache instance to store, retrieve, and delete data:

err := cache.Set("my-key", []byte("my-value"))
if err != nil {
    // handle error
}

value, err := cache.Get("my-key")
if err != nil {
    // handle error
}

err := cache.Delete("my-key")
if err != nil {
    // handle error
}

That's the basic getting started guide for using BigCache in your Go application. You can further customize the cache configuration and behavior to suit your specific needs.

Competitor Comparisons

go-cache

8,063

An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

Pros of go-cache

Simplicity: go-cache is a lightweight, in-memory cache library that is easy to use and integrate into Go projects.
Flexibility: go-cache provides a simple and flexible API that allows developers to customize the cache behavior to fit their specific needs.
Performance: go-cache is designed to be fast and efficient, with low overhead and minimal memory usage.

Cons of go-cache

Limited Features: go-cache lacks some of the more advanced features found in BigCache, such as persistence, sharding, and expiration callbacks.
Scalability: go-cache may not be as scalable as BigCache, especially for large-scale applications with high cache usage.

Code Comparison

go-cache:

cache := goCache.New(5*time.Minute, 10*time.Minute)
cache.Set("key", "value", goCache.DefaultExpiration)
value, found := cache.Get("key")

BigCache:

cache, _ := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
cache.Set("key", []byte("value"))
value, _ := cache.Get("key")

gcache

2,578

An in-memory cache library for golang. It supports multiple eviction policies: LRU, LFU, ARC

Pros of bluele/gcache

Supports multiple cache eviction policies, including LRU, LFU, and FIFO, allowing for more flexibility in cache management.
Provides a simple and intuitive API for interacting with the cache, making it easy to integrate into existing projects.
Includes support for expiration of cache entries, which can be useful for caching data that has a limited lifespan.

Cons of bluele/gcache

May have a smaller user base and community compared to the more popular BigCache project, which could mean fewer resources and support available.
Lacks some of the advanced features and optimizations found in BigCache, such as the ability to shard the cache across multiple machines.

Code Comparison

BigCache:

cache, _ := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
cache.Set("my-unique-key", []byte("value"))
value, _ := cache.Get("my-unique-key")

gcache:

cache := gcache.New(100).LRU().Expiration(10 * time.Minute).Build()
cache.Set("my-unique-key", "value")
value, _ := cache.Get("my-unique-key")

freecache

5,056

A cache library for Go with zero GC overhead.

Pros of FreeCahe

FreeCahe is designed to be a lightweight and fast in-memory cache, with a focus on simplicity and performance.
FreeCahe provides a simple and easy-to-use API, making it a good choice for developers who need a quick and efficient caching solution.
FreeCahe is written in Go, which is known for its performance and concurrency features, making it well-suited for high-throughput applications.

Cons of FreeCahe

FreeCahe may not have the same level of features and customization options as BigCache, which could be a drawback for developers with more complex caching requirements.
FreeCahe has a smaller community and ecosystem compared to BigCache, which may mean fewer resources and support available.
FreeCahe may not have the same level of maturity and stability as BigCache, which has been in development for a longer period of time.

Code Comparison

BigCache:

cache, err := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
if err != nil {
    // handle error
}

err = cache.Set("my-unique-key", []byte("value"))
if err != nil {
    // handle error
}

value, err := cache.Get("my-unique-key")
if err != nil {
    // handle error
}

FreeCahe:

cache := freecache.NewCache(100 * 1024 * 1024) // 100MB

err := cache.Set([]byte("key"), []byte("value"), 60) // 60 seconds TTL
if err != nil {
    // handle error
}

value, err := cache.Get([]byte("key"))
if err != nil {
    // handle error
}

ristretto

5,532

A high performance memory-bound Go cache

Pros of Ristretto

Ristretto provides a more advanced and feature-rich cache implementation compared to BigCache, with support for expiration, eviction policies, and more.
Ristretto is designed to be highly concurrent and scalable, with a focus on performance and efficiency.
Ristretto offers a more flexible and customizable API, allowing users to tailor the cache to their specific needs.

Cons of Ristretto

Ristretto may have a steeper learning curve compared to BigCache, as it offers more advanced features and configuration options.
Ristretto may have a higher memory footprint than BigCache, depending on the specific use case and configuration.
Ristretto is a relatively newer project compared to BigCache, and may have a smaller community and fewer resources available.

Code Comparison

BigCache:

cache := bigcache.New(bigcache.DefaultConfig(10 * time.Minute))
cache.Set("my-unique-key", []byte("value"))
value, err := cache.Get("my-unique-key")

Ristretto:

cache := ristretto.NewCache(&ristretto.Config{
    NumCounters: 1e7,     // number of keys to track frequency of (10M).
    MaxCost:     1 << 30, // maximum cost of cache (1GB).
    BufferItems: 64,      // number of keys per Get buffer.
})
cache.Set("my-unique-key", []byte("value"), 1)
value, found := cache.Get("my-unique-key")

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

BigCache

Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance. BigCache keeps entries on heap but omits GC for them. To achieve that, operations on byte slices take place, therefore entries (de)serialization in front of the cache will be needed in most use cases.

Requires Go 1.12 or newer.

Usage

Simple initialization

import (
	"fmt"
	"context"
	"github.com/allegro/bigcache/v3"
)

cache, _ := bigcache.New(context.Background(), bigcache.DefaultConfig(10 * time.Minute))

cache.Set("my-unique-key", []byte("value"))

entry, _ := cache.Get("my-unique-key")
fmt.Println(string(entry))

Custom initialization

When cache load can be predicted in advance then it is better to use custom initialization because additional memory allocation can be avoided in that way.

import (
	"log"

	"github.com/allegro/bigcache/v3"
)

config := bigcache.Config {
		// number of shards (must be a power of 2)
		Shards: 1024,

		// time after which entry can be evicted
		LifeWindow: 10 * time.Minute,

		// Interval between removing expired entries (clean up).
		// If set to <= 0 then no action is performed.
		// Setting to < 1 second is counterproductive â bigcache has a one second resolution.
		CleanWindow: 5 * time.Minute,

		// rps * lifeWindow, used only in initial memory allocation
		MaxEntriesInWindow: 1000 * 10 * 60,

		// max entry size in bytes, used only in initial memory allocation
		MaxEntrySize: 500,

		// prints information about additional memory allocation
		Verbose: true,

		// cache will not allocate more memory than this limit, value in MB
		// if value is reached then the oldest entries can be overridden for the new ones
		// 0 value means no size limit
		HardMaxCacheSize: 8192,

		// callback fired when the oldest entry is removed because of its expiration time or no space left
		// for the new entry, or because delete was called. A bitmask representing the reason will be returned.
		// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
		OnRemove: nil,

		// OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left
		// for the new entry, or because delete was called. A constant representing the reason will be passed through.
		// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
		// Ignored if OnRemove is specified.
		OnRemoveWithReason: nil,
	}

cache, initErr := bigcache.New(context.Background(), config)
if initErr != nil {
	log.Fatal(initErr)
}

cache.Set("my-unique-key", []byte("value"))

if entry, err := cache.Get("my-unique-key"); err == nil {
	fmt.Println(string(entry))
}

`LifeWindow` & `CleanWindow`

LifeWindow is a time. After that time, an entry can be called dead but not deleted.
CleanWindow is a time. After that time, all the dead entries will be deleted, but not the entries that still have life.

Benchmarks

Three caches were compared: bigcache, freecache and map. Benchmark tests were made using an i7-6700K CPU @ 4.00GHz with 32GB of RAM on Ubuntu 18.04 LTS (5.2.12-050212-generic).

Benchmarks source code can be found here

Writes and reads

go version
go version go1.13 linux/amd64

go test -bench=. -benchmem -benchtime=4s ./... -timeout 30m
goos: linux
goarch: amd64
pkg: github.com/allegro/bigcache/v3/caches_bench
BenchmarkMapSet-8                     	12999889	       376 ns/op	     199 B/op	       3 allocs/op
BenchmarkConcurrentMapSet-8           	 4355726	      1275 ns/op	     337 B/op	       8 allocs/op
BenchmarkFreeCacheSet-8               	11068976	       703 ns/op	     328 B/op	       2 allocs/op
BenchmarkBigCacheSet-8                	10183717	       478 ns/op	     304 B/op	       2 allocs/op
BenchmarkMapGet-8                     	16536015	       324 ns/op	      23 B/op	       1 allocs/op
BenchmarkConcurrentMapGet-8           	13165708	       401 ns/op	      24 B/op	       2 allocs/op
BenchmarkFreeCacheGet-8               	10137682	       690 ns/op	     136 B/op	       2 allocs/op
BenchmarkBigCacheGet-8                	11423854	       450 ns/op	     152 B/op	       4 allocs/op
BenchmarkBigCacheSetParallel-8        	34233472	       148 ns/op	     317 B/op	       3 allocs/op
BenchmarkFreeCacheSetParallel-8       	34222654	       268 ns/op	     350 B/op	       3 allocs/op
BenchmarkConcurrentMapSetParallel-8   	19635688	       240 ns/op	     200 B/op	       6 allocs/op
BenchmarkBigCacheGetParallel-8        	60547064	        86.1 ns/op	     152 B/op	       4 allocs/op
BenchmarkFreeCacheGetParallel-8       	50701280	       147 ns/op	     136 B/op	       3 allocs/op
BenchmarkConcurrentMapGetParallel-8   	27353288	       175 ns/op	      24 B/op	       2 allocs/op
PASS
ok  	github.com/allegro/bigcache/v3/caches_bench	256.257s

Writes and reads in bigcache are faster than in freecache. Writes to map are the slowest.

GC pause time

go version
go version go1.13 linux/amd64

go run caches_gc_overhead_comparison.go

Number of entries:  20000000
GC pause for bigcache:  1.506077ms
GC pause for freecache:  5.594416ms
GC pause for map:  9.347015ms

go version
go version go1.13 linux/arm64

go run caches_gc_overhead_comparison.go
Number of entries:  20000000
GC pause for bigcache:  22.382827ms
GC pause for freecache:  41.264651ms
GC pause for map:  72.236853ms

Test shows how long are the GC pauses for caches filled with 20mln of entries. Bigcache and freecache have very similar GC pause time.

Memory usage

You may encounter system memory reporting what appears to be an exponential increase, however this is expected behaviour. Go runtime allocates memory in chunks or 'spans' and will inform the OS when they are no longer required by changing their state to 'idle'. The 'spans' will remain part of the process resource usage until the OS needs to repurpose the address. Further reading available here.

How it works

BigCache relies on optimization presented in 1.5 version of Go (issue-9477). This optimization states that if map without pointers in keys and values is used then GC will omit its content. Therefore BigCache uses map[uint64]uint32 where keys are hashed and values are offsets of entries.

Entries are kept in byte slices, to omit GC again. Byte slices size can grow to gigabytes without impact on performance because GC will only see single pointer to it.

Collisions

BigCache does not handle collisions. When new item is inserted and it's hash collides with previously stored item, new item overwrites previously stored value.

Bigcache vs Freecache

Both caches provide the same core features but they reduce GC overhead in different ways. Bigcache relies on map[uint64]uint32, freecache implements its own mapping built on slices to reduce number of pointers.

Results from benchmark tests are presented above. One of the advantage of bigcache over freecache is that you donât need to know the size of the cache in advance, because when bigcache is full, it can allocate additional memory for new entries instead of overwriting existing ones as freecache does currently. However hard max size in bigcache also can be set, check HardMaxCacheSize.

HTTP Server

This package also includes an easily deployable HTTP implementation of BigCache, which can be found in the server package.

Bigcache genesis is described in allegro.tech blog post: writing a very fast cache service in Go

License

BigCache is released under the Apache 2.0 license (see LICENSE)

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

Top Related Projects

Quick Overview

Pros

Cons

Code Examples

Getting Started

Competitor Comparisons

Pros of go-cache

Cons of go-cache

Code Comparison

Pros of bluele/gcache

Cons of bluele/gcache

Code Comparison

Pros of FreeCahe

Cons of FreeCahe

Code Comparison

Pros of Ristretto

Cons of Ristretto

Code Comparison

Convert designs to code with AI

README

BigCache

Usage

Simple initialization

Custom initialization

LifeWindow & CleanWindow

Writes and reads

GC pause time

Memory usage

How it works

Collisions

Bigcache vs Freecache

HTTP Server

More

License

Top Related Projects

Convert designs to code with AI

`LifeWindow` & `CleanWindow`