scout-elasticsearch-driver

This package offers advanced functionality for searching and filtering data in Elasticsearch.

1,164

310

1,164

141

View on GitHub

Top Related Projects

elasticsearch-php

5,272

Official PHP client for Elasticsearch.

Elastica

2,256

Elastica is a PHP client for elasticsearch

Quick Overview

The babenkoivan/scout-elasticsearch-driver is a Laravel Scout driver for Elasticsearch. It provides seamless integration between Laravel's Scout search functionality and Elasticsearch, allowing developers to easily implement powerful full-text search capabilities in their Laravel applications using Elasticsearch as the backend.

Pros

Seamless integration with Laravel Scout and Elasticsearch
Supports advanced Elasticsearch features like custom analyzers and suggesters
Provides a fluent API for building complex search queries
Actively maintained with regular updates and improvements

Cons

Requires knowledge of Elasticsearch concepts and query DSL for advanced usage
May have a steeper learning curve compared to simpler search solutions
Performance can be affected by improper Elasticsearch configuration
Limited documentation for some advanced features

Code Examples

Basic search query:

$results = YourModel::search('query string')->get();

This performs a basic search on the indexed model using the provided query string.

Custom search with filters:

$results = YourModel::search('query string')
    ->where('category', 'electronics')
    ->whereBetween('price', [100, 500])
    ->get();

This example demonstrates how to add filters to the search query using familiar Laravel syntax.

Aggregations:

$results = YourModel::search('*')
    ->aggregate('average_price', [
        'avg' => [
            'field' => 'price'
        ]
    ])
    ->raw();

This example shows how to use Elasticsearch aggregations to calculate the average price of all documents.

Getting Started

Install the package via Composer:

composer require babenkoivan/scout-elasticsearch-driver

Add the service provider to your config/app.php:

'providers' => [
    // ...
    ScoutElastic\ScoutElasticServiceProvider::class,
],

Set the Scout driver in your .env file:
```
SCOUT_DRIVER=elastic
```

Configure Elasticsearch connection in config/scout.php:

'elastic' => [
    'client' => [
        'hosts' => [
            env('ELASTICSEARCH_HOST', 'localhost:9200'),
        ],
    ],
],

Make your model searchable:

use Laravel\Scout\Searchable;
use ScoutElastic\Searchable as ElasticSearchable;

class YourModel extends Model
{
    use Searchable, ElasticSearchable;
}

Competitor Comparisons

elasticsearch-php

5,272

Official PHP client for Elasticsearch.

Pros of elasticsearch-php

Official Elasticsearch PHP client, ensuring compatibility and up-to-date features
More flexible and customizable for complex Elasticsearch operations
Suitable for projects beyond Laravel, offering broader applicability

Cons of elasticsearch-php

Requires more manual configuration and setup for Laravel integration
Steeper learning curve, especially for developers new to Elasticsearch
Lacks Laravel-specific optimizations and conveniences

Code Comparison

scout-elasticsearch-driver:

use ScoutElastic\Searchable;

class Article extends Model
{
    use Searchable;
}

elasticsearch-php:

$client = ClientBuilder::create()->build();
$params = [
    'index' => 'my_index',
    'body'  => ['query' => ['match' => ['title' => 'Search']]]
];
$response = $client->search($params);

The scout-elasticsearch-driver example shows how easily a model can be made searchable using the trait. In contrast, the elasticsearch-php example demonstrates the more verbose but flexible approach to performing a search query directly with the Elasticsearch client.

scout-elasticsearch-driver is tailored for Laravel Scout, providing a smoother integration with Laravel's ORM and search functionality. It abstracts away much of the complexity of working directly with Elasticsearch, making it more accessible for Laravel developers.

elasticsearch-php, being the official client, offers more control and flexibility but requires more in-depth knowledge of Elasticsearch and additional setup for Laravel integration. It's better suited for complex Elasticsearch operations or projects not using Laravel.

Elastica

2,256

Elastica is a PHP client for elasticsearch

Pros of Elastica

More mature and established project with a longer history
Supports a wider range of Elasticsearch features and versions
Offers more flexibility and control over Elasticsearch operations

Cons of Elastica

Steeper learning curve due to its comprehensive feature set
Requires more manual configuration and setup compared to Scout Elasticsearch Driver
Less seamless integration with Laravel's Scout package

Code Comparison

Scout Elasticsearch Driver:

use App\Models\Post;

$results = Post::search('Laravel')->get();

Elastica:

use Elastica\Query;
use Elastica\Query\Match;

$query = new Query();
$match = new Match();
$match->setField('content', 'Laravel');
$query->setQuery($match);
$results = $index->search($query);

The Scout Elasticsearch Driver provides a more Laravel-friendly syntax, while Elastica offers more granular control over query construction. Scout Elasticsearch Driver is designed specifically for Laravel Scout integration, making it easier to use within Laravel applications. Elastica, being a standalone library, requires more setup but provides greater flexibility for complex Elasticsearch operations across various PHP projects.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Scout Elasticsearch Driver

ð¥ Introducing a new Elasticsearch ecosystem for Laravel. ð¥

This package offers advanced functionality for searching and filtering data in Elasticsearch. Check out its features!

Features
Requirements
Installation
Configuration
Index configurator
Searchable model
Usage
Console commands
Search rules
Available filters
Zero downtime migration
Debug
Alternatives

Features

An easy way to configure and create an Elasticsearch index.
A fully configurable mapping for each model.
A possibility to add a new field to an existing mapping automatically or using the artisan command.
Lots of different ways to implement your search algorithm: using search rules or a raw search.
Various filter types to make a search query more specific.
Zero downtime migration from an old index to a new index.
Bulk indexing, see the configuration section.

Requirements

The package has been tested in the following configuration:

PHP version >=7.1.3, <=7.3
Laravel Framework version >=5.8, <=6
Elasticsearch version >=7

Installation

Use composer to install the package:

composer require babenkoivan/scout-elasticsearch-driver

If you are using Laravel version <= 5.4 or the package discovery is disabled, add the following providers in config/app.php:

'providers' => [
    Laravel\Scout\ScoutServiceProvider::class,
    ScoutElastic\ScoutElasticServiceProvider::class,
]

Configuration

To configure the package you need to publish settings first:

php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"
php artisan vendor:publish --provider="ScoutElastic\ScoutElasticServiceProvider"

Then, set the driver setting to elastic in the config/scout.php file (or set SCOUT_DRIVER=elastic in the .env) and configure the driver itself in the config/scout_elastic.php file. The available options are:

Option	Description
client	A setting hash to build Elasticsearch client. More information you can find here. By default the host is set to `localhost:9200`.
update_mapping	The option that specifies whether to update a mapping automatically or not. By default it is set to `true`.
indexer	Set to `single` for the single document indexing and to `bulk` for the bulk document indexing. By default is set to `single`.
document_refresh	This option controls when updated documents appear in the search results. Can be set to `'true'`, `'false'`, `'wait_for'` or `null`. More details about this option you can find here. By default set to `null`.

Note, that if you use the bulk document indexing you'll probably want to change the chunk size, you can do that in the config/scout.php file.

Index configurator

An index configurator class is used to set up settings for an Elasticsearch index. To create a new index configurator use the following artisan command:

php artisan make:index-configurator MyIndexConfigurator

It'll create the file MyIndexConfigurator.php in the app folder of your project. You can specify index name and settings like in the following example:

<?php

namespace App;

use ScoutElastic\IndexConfigurator;

class MyIndexConfigurator extends IndexConfigurator
{
    // It's not obligatory to determine name. By default it'll be a snaked class name without `IndexConfigurator` part.
    protected $name = 'my_index';  
    
    // You can specify any settings you want, for example, analyzers. 
    protected $settings = [
        'analysis' => [
            'analyzer' => [
                'es_std' => [
                    'type' => 'standard',
                    'stopwords' => '_spanish_'
                ]
            ]    
        ]
    ];
}

More about index settings you can find in the index management section of Elasticsearch documentation.

To create an index just run the artisan command:

php artisan elastic:create-index "App\MyIndexConfigurator"

Note, that every searchable model requires its own index configurator.

Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. Indices created in 5.x with multiple mapping types will continue to function as before in Elasticsearch 6.x. Mapping types will be completely removed in Elasticsearch 7.0.0.

You can find more information here.

Searchable model

To create a model with the ability to perform search requests in an Elasticsearch index use the command:

php artisan make:searchable-model MyModel --index-configurator=MyIndexConfigurator

After executing the command you'll find the file MyModel.php in you app folder:

<?php

namespace App;

use ScoutElastic\Searchable;
use Illuminate\Database\Eloquent\Model;

class MyModel extends Model
{
    use Searchable;

    protected $indexConfigurator = MyIndexConfigurator::class;

    protected $searchRules = [
        //
    ];

    // Here you can specify a mapping for model fields
    protected $mapping = [
        'properties' => [
            'title' => [
                'type' => 'text',
                // Also you can configure multi-fields, more details you can find here https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html
                'fields' => [
                    'raw' => [
                        'type' => 'keyword',
                    ]
                ]
            ],
        ]
    ];
}

Each searchable model represents an Elasticsearch type. By default a type name is the same as a table name, but you can set any type name you want through the searchableAs method. You can also specify fields which will be indexed by the driver through the toSearchableArray method. More information about these options you will find in the scout official documentation.

The last important option you can set in the MyModel class is the $searchRules property. It allows you to set different search algorithms for a model. We'll take a closer look at it in the search rules section.

After setting up a mapping in your model you can update an Elasticsearch type mapping:

php artisan elastic:update-mapping "App\MyModel"

Usage

Once you've created an index configurator, an Elasticsearch index itself and a searchable model, you are ready to go. Now you can index and search data according to the documentation.

Basic search usage example:

// set query string
App\MyModel::search('phone')
    // specify columns to select
    ->select(['title', 'price'])
    // filter 
    ->where('color', 'red')
    // sort
    ->orderBy('price', 'asc')
    // collapse by field
    ->collapse('brand')
    // set offset
    ->from(0)
    // set limit
    ->take(10)
    // get results
    ->get();

If you only need the number of matches for a query, use the count method:

App\MyModel::search('phone') 
    ->count();

If you need to load relations, use the with method:

App\MyModel::search('phone') 
    ->with('makers')
    ->get();

In addition to standard functionality the package offers you the possibility to filter data in Elasticsearch without specifying a query string:

App\MyModel::search('*')
    ->where('id', 1)
    ->get();

Also you can override model search rules:

App\MyModel::search('Brazil')
    ->rule(App\MySearchRule::class)
    ->get();

And use variety of where conditions:

App\MyModel::search('*')
    ->whereRegexp('name.raw', 'A.+')
    ->where('age', '>=', 30)
    ->whereExists('unemployed')
    ->get();

And filter out results with a score less than min_score:

App\MyModel::search('sales')
    ->minScore(1.0)
    ->get();

And add more complex sorting (geo_distance eg.)

$model = App\MyModel::search('sales')
    ->orderRaw([
       '_geo_distance' =>  [
           'coordinates' => [
               'lat'   =>  51.507351,
               'lon'   =>  -0.127758
           ],
           'order'     =>  'asc',
           'unit'      =>  'm'
       ]
    ])
    ->get();

// To retrieve sort result, use model `sortPayload` attribute:
$model->sortPayload;

At last, if you want to send a custom request, you can use the searchRaw method:

App\MyModel::searchRaw([
    'query' => [
        'bool' => [
            'must' => [
                'match' => [
                    '_all' => 'Brazil'
                ]
            ]
        ]
    ]
]);

This query will return raw response.

Console commands

Available artisan commands are listed below:

Command	Arguments	Description
make:index-configurator	`name` - The name of the class	Creates a new Elasticsearch index configurator.
make:searchable-model	`name` - The name of the class	Creates a new searchable model.
make:search-rule	`name` - The name of the class	Creates a new search rule.
elastic:create-index	`index-configurator` - The index configurator class	Creates an Elasticsearch index.
elastic:update-index	`index-configurator` - The index configurator class	Updates settings and mappings of an Elasticsearch index.
elastic:drop-index	`index-configurator` - The index configurator class	Drops an Elasticsearch index.
elastic:update-mapping	`model` - The model class	Updates a model mapping.
elastic:migrate-model	`model` - The model class, `target-index` - The index name to migrate	Migrates model to another index.

For detailed description and all available options run php artisan help [command] in the command line.

Search rules

A search rule is a class that describes how a search query will be executed. To create a search rule use the command:

php artisan make:search-rule MySearchRule

In the file app/MySearchRule.php you will find a class definition:

<?php

namespace App;

use ScoutElastic\SearchRule;

class MySearch extends SearchRule
{
    // This method returns an array, describes how to highlight the results.
    // If null is returned, no highlighting will be used. 
    public function buildHighlightPayload()
    {
        return [
            'fields' => [
                'name' => [
                    'type' => 'plain'
                ]
            ]
        ];
    }
    
    // This method returns an array, that represents bool query.
    public function buildQueryPayload()
    {
        return [
            'must' => [
                'match' => [
                    'name' => $this->builder->query
                ]
            ]
        ];
    }
}

You can read more about bool queries here and about highlighting here.

The default search rule returns the following payload:

return [
   'must' => [
       'query_string' => [
           'query' => $this->builder->query
       ]
   ]
];

This means that by default when you call search method on a model it tries to find the query string in any field.

To determine default search rules for a model just add a property:

<?php

namespace App;

use ScoutElastic\Searchable;
use Illuminate\Database\Eloquent\Model;

class MyModel extends Model
{
    use Searchable;
    
    // You can set several rules for one model. In this case, the first not empty result will be returned.
    protected $searchRules = [
        MySearchRule::class
    ];
}

You can also set a search rule in a query builder:

// You can set either a SearchRule class
App\MyModel::search('Brazil')
    ->rule(App\MySearchRule::class)
    ->get();
    
// or a callable
App\MyModel::search('Brazil')
    ->rule(function($builder) {
        return [
            'must' => [
                'match' => [
                    'Country' => $builder->query
                ]
            ]
        ];
    })
    ->get();

To retrieve highlight, use model highlight attribute:

// Let's say we highlight field `name` of `MyModel`.
$model = App\MyModel::search('Brazil')
    ->rule(App\MySearchRule::class)
    ->first();

// Now you can get raw highlighted value:
$model->highlight->name;

// or string value:
 $model->highlight->nameAsString;

Available filters

You can use different types of filters:

Method	Example	Description
where($field, $value)	where('id', 1)	Checks equality to a simple value.
where($field, $operator, $value)	where('id', '>=', 1)	Filters records according to a given rule. Available operators are: =, <, >, <=, >=, <>.
whereIn($field, $value)	whereIn('id', [1, 2, 3])	Checks if a value is in a set of values.
whereNotIn($field, $value)	whereNotIn('id', [1, 2, 3])	Checks if a value isn't in a set of values.
whereBetween($field, $value)	whereBetween('price', [100, 200])	Checks if a value is in a range.
whereNotBetween($field, $value)	whereNotBetween('price', [100, 200])	Checks if a value isn't in a range.
whereExists($field)	whereExists('unemployed')	Checks if a value is defined.
whereNotExists($field)	whereNotExists('unemployed')	Checks if a value isn't defined.
whereMatch($field, $value)	whereMatch('tags', 'travel')	Filters records matching exact value. Here you can find more about syntax.
whereNotMatch($field, $value)	whereNotMatch('tags', 'travel')	Filters records not matching exact value. Here you can find more about syntax.
whereRegexp($field, $value, $flags = 'ALL')	whereRegexp('name.raw', 'A.+')	Filters records according to a given regular expression. Here you can find more about syntax.
whereGeoDistance($field, $value, $distance)	whereGeoDistance('location', [-70, 40], '1000m')	Filters records according to given point and distance from it. Here you can find more about syntax.
whereGeoBoundingBox($field, array $value)	whereGeoBoundingBox('location', ['top_left' => [-74.1, 40.73], 'bottom_right' => [-71.12, 40.01]])	Filters records within given boundings. Here you can find more about syntax.
whereGeoPolygon($field, array $points)	whereGeoPolygon('location', [[-70, 40],[-80, 30],[-90, 20]])	Filters records within given polygon. Here you can find more about syntax.
whereGeoShape($field, array $shape, $relation = 'INTERSECTS')	whereGeoShape('shape', ['type' => 'circle', 'radius' => '1km', 'coordinates' => [4, 52]], 'WITHIN')	Filters records within given shape. Here you can find more about syntax.

In most cases it's better to use raw fields to filter records, i.e. not analyzed fields.

Zero downtime migration

As you might know, you can't change the type of already created field in Elasticsearch. The only choice in such case is to create a new index with necessary mapping and import your models into the new index.
A migration can take quite a long time, so to avoid downtime during the process the driver reads from the old index and writes to the new one. As soon as migration is over it starts reading from the new index and removes the old index. This is how the artisan elastic:migrate-model command works.

Before you run the command, make sure that your index configurator uses the ScoutElastic\Migratable trait. If it's not, add the trait and run the artisan elastic:update-index command using your index configurator class name as an argument:

php artisan elastic:update-index "App\MyIndexConfigurator"

When you are ready, make changes in the model mapping and run the elastic:migrate-model command using the model class as the first argument and desired index name as the second argument:

php artisan elastic:migrate-model "App\MyModel" my_index_v2

Note, that if you need just to add new fields in your mapping, use the elastic:update-mapping command.

Debug

There are two methods that can help you to analyze results of a search query:

explain

App\MyModel::search('Brazil')
    ->explain();

profile

App\MyModel::search('Brazil')
    ->profile();

Both methods return raw data from ES.

Besides, you can get a query payload that will be sent to ES, by calling the buildPayload method.

App\MyModel::search('Brazil')
    ->buildPayload();

Note, that this method returns a collection of payloads, because of possibility of using multiple search rules in one query.

Alternatives

Recently I've released a new Elasticsearch ecosystem for Laravel, it includes:

Elastic Scout Driver - a generic Elasticsearch driver for Laravel Scout. It's perfect, if you need to build a simple search in your Laravel application.
Elastic Scout Driver Plus - an extension for Elastic Scout Driver. If you want to take advantage of such Elasticsearch features as bool queries, highlighting, etc., it's a way to go.
Elastic Migrations - an easy way to create, delete or update Elasticsearch index schema and share it with your teammates. It has quite similar interface to Laravel database migrations.

If any of it sounds interesting to you and you want to get more details, please read The Ultimate Guide to Elasticsearch in Laravel Application. The article makes a good overview of the mentioned packages and provides usage examples.

FAQ:

Why did you create a new package instead of a new scout-elasticsearch-driver version? - I didn't want to create another all in one package for obvious reasons: no separation of concerns, not compatible with other Scout drivers, hard to test and develop, etc. As Elastic Scout Driver is a generic driver and doesn't implement all the scout-elasticsearch-driver features, it would be wrong to call it a new scout-elasticsearch-driver version.
What does it mean for scout-elasticsearch-driver? - Well, it's maintained by the community at the moment (thank you @iget-esoares and @lucamichot for keeping the project alive ð). I hope they will continue contributing to the project and bring a new version of scout-elasticsearch-driver in the future.

Top Related Projects

elasticsearch-php

5,272

Official PHP client for Elasticsearch.

Elastica

2,256

Elastica is a PHP client for elasticsearch

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

Menu

scout-elasticsearch-driver

Top Related Projects

elasticsearch-php

Elastica

Quick Overview

Pros

Cons

Code Examples

Getting Started

Competitor Comparisons

elasticsearch-php

Pros of elasticsearch-php

Cons of elasticsearch-php

Code Comparison

Elastica

Pros of Elastica

Cons of Elastica

Code Comparison

Convert designs to code with AI

README

Scout Elasticsearch Driver

Contents

Features

Requirements

Installation

Configuration

Index configurator

Searchable model

Usage

Console commands

Search rules

Available filters

Zero downtime migration

Debug

Alternatives

Top Related Projects

elasticsearch-php

Elastica

Convert designs to code with AI