zoekt

Fast trigram based code search

1,692

113

1,692

View on GitHub

Top Related Projects

linguist

12,111

Language Savant. If your repository's language is being reported incorrectly, send us a pull request!

kibana

19,705

Your window into the Elastic Stack

elasticsearch

69,433

Free and Open, Distributed, RESTful Search Engine

Quick Overview

Zoekt is a fast, scalable, and distributed code search engine developed by Google. It allows users to quickly search through large codebases, making it a valuable tool for software development teams and open-source projects.

Pros

Fast and Scalable: Zoekt is designed to handle large codebases efficiently, providing fast search results even for projects with millions of files.
Distributed Architecture: The project supports a distributed architecture, allowing it to scale across multiple machines and handle high search loads.
Customizable: Zoekt offers a range of configuration options, enabling users to tailor the search engine to their specific needs.
Open-Source: The project is open-source, allowing developers to contribute, extend, and customize the codebase as needed.

Cons

Limited Documentation: The project's documentation could be more comprehensive, making it challenging for new users to get started.
Steep Learning Curve: Configuring and deploying Zoekt may require a certain level of technical expertise, which could be a barrier for some users.
Dependency on Bazel: Zoekt relies on the Bazel build system, which may not be familiar to all developers.
Lack of Graphical User Interface: Zoekt is primarily a command-line tool, which may not be preferred by all users.

Code Examples

Zoekt is a code search engine, so it does not provide a traditional code library. However, here are a few examples of how to use the Zoekt command-line interface:

Searching for a Specific Term:

zoekt-grep -i 'my_function' /path/to/repository

This command searches for the term "my_function" (case-insensitive) within the specified repository.

Searching for a Regular Expression:

zoekt-grep -r 'import\s+\w+' /path/to/repository

This command searches for all lines that match the regular expression "import \w+" within the specified repository.

Searching Across Multiple Repositories:

zoekt-server --index_dir=/path/to/index --listen=:6070
zoekt-grep -server=:6070 -i 'my_function' /path/to/repository1 /path/to/repository2

This example demonstrates how to set up a Zoekt server and use it to search across multiple repositories.

Getting Started

To get started with Zoekt, follow these steps:

Install the necessary dependencies, including Bazel and Go.
Clone the Zoekt repository from GitHub:

git clone https://github.com/google/zoekt.git

Build the Zoekt binaries:

cd zoekt
bazel build //...

Index your codebase:

bazel run //cmd/zoekt-index -- /path/to/repository -o /path/to/index

Start the Zoekt server:

bazel run //cmd/zoekt-server -- --index_dir=/path/to/index --listen=:6070

Use the zoekt-grep command to search your codebase:

bazel run //cmd/zoekt-grep -- -server=:6070 -i 'my_function'

For more detailed instructions and configuration options, please refer to the Zoekt project's documentation.

Competitor Comparisons

vscode

162,288

Visual Studio Code

Pros of VSCode

Extensive plugin ecosystem with thousands of extensions available
Highly customizable with a wide range of settings and themes
Provides a rich set of features for various programming languages and tasks

Cons of VSCode

Larger in size and resource-intensive compared to Zoekt
Slower startup time due to the extensive feature set and plugin ecosystem
May not be as lightweight and focused as Zoekt for specific use cases

Code Comparison

VSCode:

function handleFileOpen() {
  const file = dialog.showOpenDialogSync({
    properties: ['openFile']
  });
  if (file) {
    openFile(file[0]);
  }
}

Zoekt:

func (s *Server) handleSearch(w http.ResponseWriter, r *http.Request) {
  q := r.URL.Query().Get("q")
  if q == "" {
    http.Error(w, "missing query", http.StatusBadRequest)
    return
  }

  res, err := s.searcher.Search(r.Context(), q)
  if err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }

  w.Header().Set("Content-Type", "application/json")
  json.NewEncoder(w).Encode(res)
}

atom

60,150

:atom: The hackable text editor

Pros of Atom

Atom is a full-featured text editor with a wide range of plugins and customization options, making it a powerful tool for developers.
The Atom community is large and active, providing a wealth of resources and support for users.
Atom has a user-friendly interface and is easy to navigate, even for beginners.

Cons of Atom

Atom can be resource-intensive, especially on older or less powerful machines, which can lead to performance issues.
The development of Atom has slowed down in recent years, with fewer updates and new features being added.
Atom may not be as efficient as some other text editors for certain tasks, such as large-scale code searching and indexing.

Code Comparison

Atom:

const editor = atom.workspace.getActiveTextEditor();
if (editor) {
  const selectedText = editor.getSelectedText();
  console.log(`Selected text: ${selectedText}`);
}

Zoekt:

func (s *Server) Search(ctx context.Context, req *zoekt.SearchRequest) (*zoekt.SearchResponse, error) {
    resp := &zoekt.SearchResponse{}
    resp.Files = make([]*zoekt.FileMatch, 0, len(s.index.Repos))
    for _, repo := range s.index.Repos {
        match, err := s.index.Search(ctx, req, repo)
        if err != nil {
            return nil, err
        }
        resp.Files = append(resp.Files, match.Files...)
    }
    return resp, nil
}

linguist

12,111

Language Savant. If your repository's language is being reported incorrectly, send us a pull request!

Pros of Linguist

Linguist is a widely used and well-established library for language detection and file type identification, with support for a large number of programming languages.
Linguist provides a comprehensive set of features, including the ability to detect programming languages, markup languages, and other file types.
Linguist is actively maintained and has a large community of contributors, ensuring ongoing development and support.

Cons of Linguist

Linguist may be more complex and feature-rich than some users require, potentially making it overkill for simpler use cases.
Linguist's dependency on external libraries and resources may make it more difficult to deploy or integrate into certain environments.

Code Comparison

Linguist:

def detect_filename(filename)
  language = nil
  filename.scan(/\.([^.]+)$/) do |extension|
    language = Language.find_by_extension(extension.first)
    break if language
  end
  language
end

Zoekt:

func (s *Server) handleSearch(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    q := r.URL.Query().Get("q")
    if q == "" {
        http.Error(w, "missing query", http.StatusBadRequest)
        return
    }

    res, err := s.searcher.Search(ctx, q)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(res)
}

kibana

19,705

Your window into the Elastic Stack

Pros of Kibana

Kibana provides a comprehensive data visualization and exploration platform, allowing users to create custom dashboards and visualizations.
The tool offers a wide range of built-in visualization types, including graphs, charts, and maps, making it easier to analyze and present data.
Kibana integrates seamlessly with the Elasticsearch search and analytics engine, providing a powerful and flexible data analysis solution.

Cons of Kibana

Kibana can be more complex to set up and configure compared to Zoekt, which has a simpler and more lightweight architecture.
The tool may have a steeper learning curve for users who are not familiar with Elasticsearch and data visualization concepts.
Kibana's performance can be affected by the size and complexity of the data being analyzed, especially in large-scale deployments.

Code Comparison

Zoekt (Google):

func (s *Server) handleSearch(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    q := r.URL.Query()
    query := q.Get("q")
    if query == "" {
        http.Error(w, "missing query", http.StatusBadRequest)
        return
    }

    res, err := s.searcher.Search(ctx, query)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    if err := json.NewEncoder(w).Encode(res); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
    }
}

Kibana (Elastic):

export function createSearchRoute(deps) {
  const { router, config, uiSettings, callCluster } = deps;

  router.get('/api/search', async (context, request, response) => {
    const { index, query, sort, from, size } = request.query;

    try {
      const result = await callCluster('search', {
        index,
        body: {
          query,
          sort,
          from: parseInt(from, 10),
          size: parseInt(size, 10),
        },
      });

      return response.ok({
        body: result,
      });
    } catch (error) {
      return response.internalError({ body: error });
    }
  });

  return router;
}

elasticsearch

69,433

Free and Open, Distributed, RESTful Search Engine

Pros of Elasticsearch

Elasticsearch is a highly scalable and distributed search engine, capable of handling large amounts of data and high-traffic applications.
Elasticsearch provides a rich set of features, including full-text search, analytics, and real-time data processing.
Elasticsearch has a large and active community, with a wide range of plugins and integrations available.

Cons of Elasticsearch

Elasticsearch can be more complex to set up and configure compared to Zoekt, which is a simpler and more lightweight search solution.
Elasticsearch may have a higher resource footprint, requiring more memory and CPU resources compared to Zoekt.

Code Comparison

Zoekt:

func (s *Server) Search(ctx context.Context, req *zoekt.SearchRequest) (*zoekt.SearchResponse, error) {
    resp := &zoekt.SearchResponse{}
    resp.Files = make([]*zoekt.FileMatch, 0, len(s.files))
    for _, f := range s.files {
        if f.Match(req) {
            resp.Files = append(resp.Files, f)
        }
    }
    return resp, nil
}

Elasticsearch:

public SearchResponse search(SearchRequest searchRequest) {
    try {
        return client.search(searchRequest, RequestOptions.DEFAULT);
    } catch (IOException e) {
        throw new ElasticsearchException("Error executing search", e);
    }
}

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

This is a fast text search engine, intended for use with source code. (Pronunciation: roughly as you would pronounce "zooked" in English)

NOTICE: github.com/sourcegraph/zoekt is the active main repository for Zoekt development.

INSTRUCTIONS

Downloading:

go get github.com/google/zoekt/

Indexing:

go install github.com/google/zoekt/cmd/zoekt-index
$GOPATH/bin/zoekt-index .

Searching

go install github.com/google/zoekt/cmd/zoekt
$GOPATH/bin/zoekt 'ngram f:READ'

Indexing git repositories:

go install github.com/google/zoekt/cmd/zoekt-git-index
$GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .

Indexing repo repositories:

go install github.com/google/zoekt/cmd/zoekt-{repo-index,mirror-gitiles}
zoekt-mirror-gitiles -dest ~/repos/ https://gfiber.googlesource.com
zoekt-repo-index \
   -name gfiber \
   -base_url https://gfiber.googlesource.com/ \
   -manifest_repo ~/repos/gfiber.googlesource.com/manifests.git \
   -repo_cache ~/repos \
   -manifest_rev_prefix=refs/heads/ --rev_prefix= \
   master:default_unrestricted.xml

Starting the web interface

go install github.com/google/zoekt/cmd/zoekt-webserver
$GOPATH/bin/zoekt-webserver -listen :6070

A more organized installation on a Linux server should use a systemd unit file, eg.

[Unit]
Description=zoekt webserver

[Service]
ExecStart=/zoekt/bin/zoekt-webserver -index /zoekt/index -listen :443  --ssl_cert /zoekt/etc/cert.pem   --ssl_key /zoekt/etc/key.pem
Restart=always

[Install]
WantedBy=default.target

SEARCH SERVICE

Zoekt comes with a small service management program:

go install github.com/google/zoekt/cmd/zoekt-indexserver

cat << EOF > config.json
[{"GithubUser": "username"},
 {"GithubOrg": "org"},
 {"GitilesURL": "https://gerrit.googlesource.com", "Name": "zoekt" }
]
EOF

$GOPATH/bin/zoekt-server -mirror_config config.json

This will mirror all repos under 'github.com/username', 'github.com/org', as well as the 'zoekt' repository. It will index the repositories.

It takes care of fetching and indexing new data and cleaning up logfiles.

The webserver can be started from a standard service management framework, such as systemd.

SYMBOL SEARCH

It is recommended to install Universal ctags to improve ranking. See here for more information.

ACKNOWLEDGEMENTS

Thanks to Alexander Neubeck for coming up with this idea, and helping me flesh it out.

DISCLAIMER

This is not an official Google product

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot