v5.0.0-RC2

This second release candidate of V5 brings in Laravel 12 compatibility along with a few bug fixes

What's Changed

• Small Bug Fixes found in RC1 by @use-the-fork in #60

Full Changelog: v5.0.0-RC1...v5.0.0-RC2

v5.0.0-RC1

This is the first release candidate of V5, a near-complete rewrite of the Laravel-Elasticsearch package. This update delivers a more seamless integration with Laravel’s Models, Eloquent, and Query Builders while providing greater access to Elasticsearch’s advanced features.

V5 introduces significant breaking changes, major upgrades, and new functionality. A full breakdown will be available in the updated documentation upon the final release.

The current RC is compatible with Laravel 10 and 11, while the final 5.0.0 release will support Laravel 12 as well.

Special thanks to @use-the-fork for initiating this rebuild and laying the groundwork (and then some) for this major release.

What's Changed

• V5.0.0 by @use-the-fork in #54

Full Changelog: v4.5.1...v5.0.0-RC1

v4.5.1

This release is compatible with Laravel 10 and 11

What's Changed

• Add updateOrCreate and updateOrCreateWithoutRefresh methods by @IonutRusen in #57
• refactor: update the return type of getConnection() by @shalomabitan in #58

New Contributors

• @IonutRusen made their first contribution in #57
• @shalomabitan made their first contribution in #58

Full Changelog: v4.5.0...v4.5.1

v4.5.0

This release is for compatibility with both Laravel 10 and 11, and marks a version bump to 4.5.0.

New features

1. Bypass field map validation for queries that require keyword mapping

Keyword type queries are checked by default and will select the keyword sub-mapping if it is found; however, this invokes an extra query to check the mapping first.

You can now disable this by setting options.bypass_map_validation = true

'elasticsearch' => [
    ......
     'options' => [
        'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
         .....
    ],
    .....
],

2. Adjustable chunk size for bulk insert (default 1000)

When performing bulk inserts, the default is 1000 at a time.

You can now adjust this by setting options.insert_chunk_size to your desired amount.

'elasticsearch' => [
    ......
    'options' => [
        'insert_chunk_size'  => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
         .....
    ],
    .....
],

Updated connection config

'elasticsearch' => [
    'driver' => 'elasticsearch',
    'auth_type' => env('ES_AUTH_TYPE', 'http'), //http or cloud
    'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
    'username' => env('ES_USERNAME', ''),
    'password' => env('ES_PASSWORD', ''),
    'cloud_id' => env('ES_CLOUD_ID', ''),
    'api_id' => env('ES_API_ID', ''),
    'api_key' => env('ES_API_KEY', ''),
    'ssl_cert' => env('ES_SSL_CA', ''),
    'ssl' => [
        'cert' => env('ES_SSL_CERT', ''),
        'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
        'key' => env('ES_SSL_KEY', ''),
        'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
    ],
    'index_prefix' => env('ES_INDEX_PREFIX', false),
    'options' => [
        'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
        'insert_chunk_size' => env('ES_OPT_INSERT_CHUNK_SIZE', 1000),
        'logging' => env('ES_OPT_LOGGING', false),
        'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
        'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
        'retires' => env('ES_OPT_RETRIES', null),
        'meta_header' => env('ES_OPT_META_HEADERS', true),
    ],
    'error_log_index' => env('ES_ERROR_INDEX', false),
],

3. Removed redundant methods, new exceptions and code clean by @use-the-fork via #48

Full Changelog: v4.4.1...v4.5.0

v4.4.1

Bug fix

• Dynamic indices field map check - issue: #47

Full Changelog: v4.4.0...v4.4.1

v4.4.0

This release introduces compatibility with both Laravel 10 and 11, and marks a version bump to 4.4.0.

While there are no breaking changes, the connection class has been fully rebuilt by @use-the-fork. This foundational improvement justifies the version increment and sets the stage for further enhancements.

What's Changed

• Refract/connection by @use-the-fork in #46

Full Changelog: v4.3.0...v4.4.0

v4.3.0

This release marks a version bump within the 4.x branch due to one breaking change in rawSearch(). This branch is committed to Laravel 10 and 11 compatibility.

Breaking Change

rawSearch($bodyParams, $returnRaw = false) has now been split into rawSearch($bodyParams) and rawDsl($bodyParams) where:

• rawSearch($bodyParams) returns an ElasticCollection of results
• rawDsl($bodyParams) returns the result body as is from Elasticsearch. Equivalent to $returnRaw = true previously.

New features

[1] Schema field mapping call: getFieldMapping(string|array $field = '*', $raw = false)

Schema method that can be called from your model:

Product::getFieldMapping('color'); //Returns a key/value array of field/types for color
Product::getFieldMapping('color',true); //Returns the mapping for  color field as is from Elasticsearch
Product::getFieldMapping(['color','name']); //Returns maapings for  color and name
Product::getFieldMapping(); //returns all field mappings, same as getFieldMapping('*')

or via Schema: Schema::getFieldMapping($index, $field, $raw)

Schema::getFieldMapping('products','color',true); 

[2] Order By Random: orderByRandom(string $column, int $seed = 1)

Uses ES's random_score to randomise ordering

Product::where('color', 'silver')->orderByRandom('created_at', rand(1, 1000))->limit(5)->get();

The value of $seed will return the same results if unchanged. This is required for consistencey, ex: pagination

Bug fix

whereExact() keyword field validator has been fixed

PRs

• Refract/code coverage - Coverage up to 79% by @use-the-fork in #40
• fix(DSL): improve keyword mapping handling for nested fields by @use-the-fork in #43
• feat(elasticsearch): enhance raw search functionality by @use-the-fork in #44

Full Changelog: v4.2.0...v4.3.0

v4.2.0

This release marks a version bump within the 4.x branch committed to Laravel 10 & 11 compatibility. There are no breaking changes from 4.1.x

This version introduces query-building methods to allow matching across multiple fields: ES docs

The current search builder works in isolation for full-text searching; this upgrade brings those features into the standard query builder that will run like a normal query. Meaning you can:
get(), first(), aggregate(), paginate() etc on full text search results. In time, this will replace the current search methods like: Book::term('Eric')->search();

Methods

searchTerm($term, $fields = ['*'], $options = []) - type: best_fields
searchTermMost($term, $fields = ['*'], $options = []) type: most_fields
searchTermCross($term, $fields = ['*'], $options = []) type: cross_fields
searchPhrase($phrase, $fields = ['*'], $options = []) type: phrase
searchPhrasePrefix($phrase, $fields = ['*'], $options = []) type: phrase_prefix
searchBoolPrefix($phrase, $fields = ['*'], $options = []) type: bool_prefix

Each method has a corresponding OR version, ex orSearchPhrase('Laravel Elasticsearch')

These methods will introduce an Elasticsearch score and will be ordered by score by default.

$fields: By default, all fields will be searched through; you can specify which as well as boost certain fields, ex:

People:searchTerm('John',['name^3','description^2','friends.name'])->get();

$options: Allows you to set any options for the multi_match clause to use, ex:
analyzer, boost, operator, minimum_should_match, fuzziness, lenient, prefix_length, max_expansions, fuzzy_rewrite, zero_terms_query.

searchFor($value) is a convenience method that will either use term or phrase depending on the word count of $value

withHighlights($fields = [], $preTag = '<em>', $postTag = '</em>', $globalOptions = [])

• Will highlight the same way as https://elasticsearch.pdphilip.com/full-text-search#highlighting

Option helpers

asFuzzy()

• Will mark the previous clause as fuzzy

setMinShouldMatch(int $value)

• will set the option {"minimum_should_match": $value} to the previous clause

setBoost(int $value)

• will set the option {"boost": $value} to the previous clause

Examples

Product::searchTerm('remarkble')->asFuzzy()->withHighlights()->get();
Product::searchPhrasePrefix('coffee bea')->where('is_active',true)->paginate();
Product::searchPhrase('United States')->orSearchPhrase('United Kingdom')->sum('orders');

Upgrades

find($id), findOrFail($id) and findOrNew($id) now uses the ES get call directly ie: /my_index/_doc/my_id

• With findOrNew($id): If the $id was not found, then it will return a new model instance with the $id value as provided
• Credit @use-the-fork via #41

Full Changelog: v4.1.1...v4.2.0

v4.1.1

Bug fix

• Bulk insert now returns _id in the error payload

Full Changelog: v4.1.0...v4.1.1

v4.1.0

V4.1.0 Release Notes

This release marks a version bump within the 4.x branch committed to Laravel 10 & 11 compatibility.

Acknowledgements

This release represents a significant advancement in the package's development, primarily driven by PR #32. Key improvements include:

• Comprehensive refactoring of the package
• Introduction of new features
• Extensive coverage with automated tests
• Enhanced compliance with coding standards and type-checking

A sincere thank you to @use-the-fork for their substantial contributions to this release. Their efforts have been instrumental in elevating the quality, reliability and maturity of this package.

Breaking Changes

There's only one breaking change:

$query->orderBy(string $column, string $direction = 'asc', string $mode = null, array $missing = '_last');
// Has been changed back to:
$query->orderBy(string $column, string $direction = 'asc');
// With a new method to add any ES sort parameter to a given column:
$query->withSort(string $col, string  $key, mixed $value);

• This also applies to orderByDesc

Examples

$products = Product::orderBy('color.keyword', 'desc', null, '_first')->get();
// Is now:
$products = Product::orderBy('color.keyword', 'desc')->withSort('color.keyword', 'missing', '_first')->get();

And:

$products = Product::where('is_active', true)->orderBy('order_values', 'desc', 'avg')->get();
// Is now:
$products = Product::where('is_active', true)->orderBy('order_values', 'desc')->withSort('order_values', 'mode', 'avg')->get();

This change is to future-proof for the outlier sorting options that ES offers, ex:

$products = Product::withSort('price', 'unmapped_type', 'long')->get();
$products = Product::withSort('price', 'ignore_unmapped', true')->get();

Inspired by #36 via @ildar-developer

Refactor

• Automated tests
• PHPStan compliance
• PINT formatted
• Updated PHPDocs for better IDE support

New Features

1. Cursor Pagination

Given the default limitations of Elasticsearch to paginate beyond 10k records, cursor pagination allows you to paginate indefinitely using search_after given a sort map.

As is the case with core Laravel's cursorPaginate , you can only page next and prev

cursorPaginate($perPage, $columns, $cursorName, $cursor)

Example:

Product::orderByDesc('orders')->orderByDesc('price')->cursorPaginate(50)->withQueryString();

The accuracy of this method depends on the sort, as would be the case using ES directly. If no sort is provided, it will try to sort by created_at and updated_at if those mappings are found; otherwise, it will throw a MissingOrderException.

---

2. Bulk Import

insert($values, $returnData = null);
insertWithoutRefresh($values, $returnData = null);

These methods use Elasticsearch's Bulk API under the hood, providing efficient ways to insert multiple documents at once.

If the $returnData parameter is null or false, it will return a summary as an array:

{
    "success": true,
    "timed_out": false,
    "took": 7725,
    "total": 100000,
    "created": 100000,
    "modified": 0,
    "failed": 0
}

Otherwise, it will return an ElasticCollection of all the inserted records.

Performance and Usage:

1. insert($values, $returnData = null)
   • Performs a bulk insert and waits for the index to refresh.
   • Ensures that inserted documents are immediately available for search.
   • Use this when you need the inserted data to be searchable right away.
   • Slower than insertWithoutRefresh but provides immediate consistency.
2. insertWithoutRefresh($values, $returnData = null)
   • Executes bulk inserts without waiting for the index to refresh.
   • Offers a significant speed boost compared to insert().
   • The speed increase is due to skipping the index refresh operation, which can be resource-intensive.
   • Inserted records may not be immediately available for search
   • Use this when you need to insert large amounts of data quickly and can tolerate a slight delay in searchability.

When to use each:

1. Use insert() when:
   • You need immediate searchability of inserted data.
   • You're inserting smaller batches of data where the performance difference is negligible.
   • In user-facing applications where real-time data availability is crucial.
2. Use insertWithoutRefresh() when:
   • You're performing large batch imports where speed is a priority.
   • In background jobs or data migration scripts where immediate searchability isn't necessary.
   • You can handle a delay between insertion and searchability in your application logic.

---

3. ElasticCollection

Queries that return collections (get(), search(), insert()) now return them as an ElasticCollection. ElasticCollection is the same as Laravel's Eloquent Collection but with the executed query's metadata embedded in it.

$fetch = Product::where('orders', '>', 100)->limit(50)->orderByDesc('price')->get();
$shards = $fetch->getShards(); // Shards info
$dsl = $fetch->getDsl(); // The full query DSL that was used
$total = $fetch->getTotal(); // Total records in the index if there were hits
$maxScore = $fetch->getMaxScore(); // Max score of the search
$took = $fetch->getTook(); // Time taken for the search in milliseconds
$meta = $fetch->getQueryMetaAsArray(); // All the metadata in an array

return $fetch; //gives your collection of results as always

Bug fix

• Starting a new query before an existing one has been executed was causing the connection (and queries) to get mixed up. via #36

What's Changed

• Refract/cleanup - Added Test suite / formating etc. by @use-the-fork in #32

Full Changelog: v4.0.4...v4.1.0