bleve

A modern text/numeric/geo-spatial/vector indexing library for go

9,971

676

9,971

301

View on GitHub

Top Related Projects

elasticsearch

69,881

Free and Open Source, Distributed, RESTful Search Engine

zincsearch

16,887

ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.

typesense

20,865

Open Source alternative to Algolia + Pinecone and an Easier-to-Use alternative to ElasticSearch ⚡ 🔍 ✨ Fast, typo tolerant, in-memory fuzzy Search Engine for building delightful search experiences

Quick Overview

Bleve is a modern text indexing library for Go. It provides full-text search capabilities with support for various languages, custom analyzers, and faceted search. Bleve is designed to be easy to use while offering powerful features for advanced search applications.

Pros

Easy to integrate with Go applications
Supports multiple languages and custom analyzers
Provides faceted search and highlighting capabilities
Offers flexible query types and scoring options

Cons

Performance may be slower compared to some other search engines
Limited documentation for advanced features
Requires more memory compared to simpler search solutions
May have a steeper learning curve for complex use cases

Code Examples

Creating an index and indexing a document:

import (
    "github.com/blevesearch/bleve/v2"
)

// Create a new index
index, _ := bleve.New("example.bleve", bleve.NewIndexMapping())

// Index a document
doc := struct {
    ID   string
    Name string
    Body string
}{
    ID:   "doc1",
    Name: "First document",
    Body: "This is the content of the first document.",
}
index.Index("doc1", doc)

Performing a search:

// Search for documents
query := bleve.NewMatchQuery("content")
searchRequest := bleve.NewSearchRequest(query)
searchResults, _ := index.Search(searchRequest)

// Print results
for _, hit := range searchResults.Hits {
    fmt.Printf("ID: %s, Score: %f\n", hit.ID, hit.Score)
}

Using faceted search:

// Add a facet field to the index mapping
indexMapping := bleve.NewIndexMapping()
englishTextFieldMapping := bleve.NewTextFieldMapping()
englishTextFieldMapping.Analyzer = "en"
indexMapping.DefaultMapping.AddFieldMappingsAt("category", englishTextFieldMapping)

// Create a search request with facets
searchRequest := bleve.NewSearchRequest(bleve.NewMatchAllQuery())
searchRequest.AddFacet("categories", bleve.NewFacetRequest("category", 5))

// Perform the search
searchResults, _ := index.Search(searchRequest)

// Print facet results
for _, facet := range searchResults.Facets["categories"].Terms {
    fmt.Printf("Category: %s, Count: %d\n", facet.Term, facet.Count)
}

Getting Started

To start using Bleve in your Go project:

Install Bleve:

go get -u github.com/blevesearch/bleve/v2

Import Bleve in your Go code:

import "github.com/blevesearch/bleve/v2"

Create an index and start indexing documents:

index, _ := bleve.New("example.bleve", bleve.NewIndexMapping())
index.Index("docID", yourDocument)

Perform searches:

query := bleve.NewMatchQuery("search terms")
searchRequest := bleve.NewSearchRequest(query)
searchResults, _ := index.Search(searchRequest)

Competitor Comparisons

elasticsearch

69,881

Free and Open Source, Distributed, RESTful Search Engine

Pros of Elasticsearch

More mature and feature-rich, with advanced functionalities like machine learning and anomaly detection
Highly scalable and distributed, suitable for large-scale enterprise deployments
Extensive ecosystem with various plugins and integrations

Cons of Elasticsearch

Higher resource requirements and complexity, potentially overkill for smaller projects
Steeper learning curve and more challenging to set up and maintain
Licensed under Elastic License 2.0, which may have limitations for some use cases

Code Comparison

Elasticsearch (Java):

IndexResponse response = client.index(new IndexRequest("posts")
    .id("1")
    .source(jsonBuilder()
        .startObject()
            .field("title", "Elasticsearch Basics")
            .field("content", "Learn about Elasticsearch...")
        .endObject()
    )
);

Bleve (Go):

index, _ := bleve.New("example.bleve", bleve.NewIndexMapping())
doc := struct {
    Title   string
    Content string
}{Title: "Bleve Basics", Content: "Learn about Bleve..."}
index.Index("1", doc)

Both Elasticsearch and Bleve are powerful search engines, but they cater to different scales and use cases. Elasticsearch is more suitable for large-scale, distributed environments with complex requirements, while Bleve offers a simpler, lightweight solution for Go applications with embedded search needs.

zincsearch

16,887

ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.

Pros of ZincSearch

Built-in web UI for easy management and visualization
Designed for high performance and scalability out of the box
Supports multiple indices and sharding

Cons of ZincSearch

Less mature and battle-tested compared to Bleve
Smaller community and ecosystem
More opinionated and less flexible for custom use cases

Code Comparison

ZincSearch (Go):

index, _ := zincsearch.NewIndex("myindex")
doc := map[string]interface{}{
    "id":   "1",
    "name": "John Doe",
}
index.Index(doc["id"].(string), doc)

Bleve (Go):

mapping := bleve.NewIndexMapping()
index, _ := bleve.New("myindex", mapping)
doc := struct {
    ID   string
    Name string
}{ID: "1", Name: "John Doe"}
index.Index(doc.ID, doc)

Both ZincSearch and Bleve are full-text search engines written in Go. ZincSearch focuses on providing a complete, ready-to-use search solution with built-in features like a web UI and high scalability. Bleve, on the other hand, offers more flexibility and customization options, making it suitable for integration into various applications and use cases. ZincSearch may be easier to set up and use for simple search needs, while Bleve provides more control over the indexing and search process for advanced requirements.

typesense

20,865

Pros of Typesense

Faster indexing and search performance, especially for large datasets
Built-in typo tolerance and fuzzy search capabilities
Simpler setup and configuration process

Cons of Typesense

Less flexible schema design compared to Bleve
Smaller community and ecosystem of plugins/extensions
Limited language support for indexing (primarily focused on English)

Code Comparison

Typesense (search query):

searchParameters := &api.SearchParameters{
    Q:        "search query",
    QueryBy:  "title,description",
    FilterBy: "category:=books",
    SortBy:   "popularity:desc",
}

Bleve (search query):

query := bleve.NewMatchQuery("search query")
searchRequest := bleve.NewSearchRequest(query)
searchRequest.Fields = []string{"title", "description"}
searchRequest.SortBy([]string{"-popularity"})

Both Typesense and Bleve offer powerful search capabilities, but Typesense focuses on simplicity and out-of-the-box performance, while Bleve provides more flexibility and customization options. Typesense is better suited for projects requiring quick setup and fast performance, whereas Bleve is ideal for applications needing fine-grained control over indexing and searching processes.

Toshi

4,181

A full-text search engine in rust

Pros of Toshi

Written in Rust, potentially offering better performance and memory safety
Designed as a distributed search engine, suitable for larger-scale applications
Supports real-time indexing and querying

Cons of Toshi

Less mature and less widely adopted compared to Bleve
Smaller community and fewer resources available
May have a steeper learning curve for developers not familiar with Rust

Code Comparison

Toshi (Rust):

let index = Index::create("my_index", IndexOptions::default())?;
index.add_document(doc!{
    "id" => 1,
    "title" => "Example Document",
    "content" => "This is a sample document for indexing."
})?;

Bleve (Go):

index, _ := bleve.New("example.bleve", bleve.NewIndexMapping())
doc := struct {
    ID      string
    Title   string
    Content string
}{ID: "1", Title: "Example Document", Content: "This is a sample document for indexing."}
index.Index("1", doc)

Both examples demonstrate creating an index and adding a document, but Toshi uses Rust's syntax and error handling, while Bleve uses Go's approach.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

bleve

A modern indexing library in GO

Features

Index any go data structure (including JSON)
Intelligent defaults backed up by powerful configuration
Supported field types:
- text, number, datetime, boolean, geopoint, geoshape, IP, vector
Supported query types:
- Term, Phrase, Match, Match Phrase, Prefix, Fuzzy
- Conjunction, Disjunction, Boolean (must/should/must_not)
- Term Range, Numeric Range, Date Range
- Geo Spatial
- Simple query string syntax
- Approximate k-nearest neighbors over vectors
tf-idf Scoring
Query time boosting
Search result match highlighting with document fragments
Aggregations/faceting support:
- Terms Facet
- Numeric Range Facet
- Date Range Facet

Indexing

message := struct{
	Id   string
	From string
	Body string
}{
	Id:   "example",
	From: "marty.schoch@gmail.com",
	Body: "bleve indexing is easy",
}

mapping := bleve.NewIndexMapping()
index, err := bleve.New("example.bleve", mapping)
if err != nil {
	panic(err)
}
index.Index(message.Id, message)

Querying

index, _ := bleve.Open("example.bleve")
query := bleve.NewQueryStringQuery("bleve")
searchRequest := bleve.NewSearchRequest(query)
searchResult, _ := index.Search(searchRequest)

Command Line Interface

To install the CLI for the latest release of bleve, run:

$ go install github.com/blevesearch/bleve/v2/cmd/bleve@latest

$ bleve --help
Bleve is a command-line tool to interact with a bleve index.

Usage:
  bleve [command]

Available Commands:
  bulk        bulk loads from newline delimited JSON files
  check       checks the contents of the index
  count       counts the number documents in the index
  create      creates a new index
  dictionary  prints the term dictionary for the specified field in the index
  dump        dumps the contents of the index
  fields      lists the fields in this index
  help        Help about any command
  index       adds the files to the index
  mapping     prints the mapping used for this index
  query       queries the index
  registry    registry lists the bleve components compiled into this executable
  scorch      command-line tool to interact with a scorch index

Flags:
  -h, --help   help for bleve

Use "bleve [command] --help" for more information about a command.

Text Analysis

Bleve includes general-purpose analyzers (customizable) as well as pre-built text analyzers for the following languages:

Arabic (ar), Bulgarian (bg), Catalan (ca), Chinese-Japanese-Korean (cjk), Kurdish (ckb), Danish (da), German (de), Greek (el), English (en), Spanish - Castilian (es), Basque (eu), Persian (fa), Finnish (fi), French (fr), Gaelic (ga), Spanish - Galician (gl), Hindi (hi), Croatian (hr), Hungarian (hu), Armenian (hy), Indonesian (id, in), Italian (it), Dutch (nl), Norwegian (no), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Swedish (sv), Turkish (tr)

Text Analysis Wizard

bleveanalysis.couchbase.com

Discussion/Issues

Discuss usage/development of bleve and/or report issues here:

License

Apache License Version 2.0

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot