Skip to content
Laravel-Opensearch

Search is only available in production builds. Try building and previewing the site to test it out locally.

Upgrading From v4

Breaking Changes

Connection

Update Index Prefix

Index Prefix no longer auto concatenates with _

   OS_INDEX_PREFIX=my_prefix
OS_INDEX_PREFIX=my_prefix_

Models

Model id Field

$model->_id is now $model->id and is an alias for OpenSearch’s _id field. This is to maintain consistency with Laravel’s Eloquent ORM.

If you were using a distinct id field alongside the default _id field then this will no longer work. You will need to update your code to use a different field name for your previous id field.

MAX_SIZE → $defaultLimit

const MAX_SIZE has been replaced with protected $defaultLimit for defining the default limit for search results.

use PDPhilip\OpenSearch\Eloquent\Model;


class Product extends Model
{
    const MAX_SIZE = 10000;
    protected $defaultLimit = 10000;
    protected $connection = 'opensearch';

Queries

where() clause now runs a term query

The where() clause now runs a term query, not a match as before.

If you still want to run a match query, then replace where() with whereMatch()

   where('name', 'John')
whereMatch('name', 'John')

If exact matching is a better fit, then ensure the field has a keyword mapping.

$index->keyword('name'); //or
$index->text('name', hasKeyword: true);

Full text search options replaced

asFuzzy(), setMinShouldMatch(), setBoost() search methods been removed. Use $options instead.

   Product::searchTerm('espreso tme')
        ->asFuzzy()->setMinShouldMatch(2)->setBoost(2)
        ->get();
Product::searchTerm('espreso tme', function (SearchOptions $options) {
    $options->searchFuzzy();
    $options->boost(2);
    $options->minimumShouldMatch(2);
})->get();

Legacy search query builder methods removed

All {xx}->search() methods have been removed. Use {multi_match}->get() instead.

   Book::term('Eric')->orTerm('Lean')->andTerm('Startup')->search();
Book::searchTerm('Eric')->orSearchTerm('Lean')->searchTerm('Startup')->get();
   Book::phrase('United States')->orPhrase('United Kingdom')->search();
Book::searchPhrase('United States')->orSearchPhrase('United Kingdom')->get();
   $results = Book::term('Eric')->fields(['title', 'author', 'description'])->search();
Book::searchTerm('Eric',['title', 'author', 'description'])->get();
   $results = Book::fuzzyTerm('quikc')->orFuzzyTerm('brwn')->andFuzzyTerm('foks')->search();
Book::searchTerm('quikc',['fuzziness' => 'AUTO'])
    ->orSearchTerm('brwn',['fuzziness' => 'AUTO'])
    ->searchTerm('foks',['fuzziness' => 'AUTO'])
    ->get();

Distinct & GroupBy

distinct() and groupBy() operate differently

The distinct() and groupBy() methods have been rebuilt and work differently.

  • distinct() uses: Nested Term Aggs

    • Returns full results, and cannot paginate
    • Sort on column names and by doc_count (useful for ordering by most aggregated values)

  • groupBy() uses: Composite Aggregation

    • Can paginate
    • Sort on column names only

Important: distinct() now executes the aggregation:

 UserLog::where('created_at', '>=', Carbon::now()->subDays(30))
    ->distinct()->get('user_id');
UserLog::where('created_at', '>=', Carbon::now()->subDays(30))
    ->distinct('user_id');

Please review the Distinct & GroupBy documentation carefully for more information.

Schema: Migration methods

IndexBlueprint/AnalyzerBlueprint → Blueprint

IndexBlueprint and AnalyzerBlueprint have been removed and replaced with a single Blueprint class

   use PDPhilip\OpenSearch\Schema\IndexBlueprint;
   use PDPhilip\OpenSearch\Schema\AnalyzerBlueprint;
use PDPhilip\OpenSearch\Schema\Blueprint;

Schema: Builder methods

Schema::hasIndex → Schema::indexExists

Schema::hasIndex has been removed. Use Schema::hasTable or Schema::indexExists instead.

   Schema::hasIndex('index_name');
Schema::hasTable('index_name'); //or
Schema::indexExists('index_name');

Schema::dsl → Schema::indices()

Schema::dsl($method,$dsl) has been removed. Use Schema::indices()->{method}

   Schema::dsl('close', ['index' => 'my_index']);
Schema::indices()->close(['index' => 'my_index']);

Schema: Blueprint index methods

  1. geo($field) field property has been replaced with geoPoint($field); 
       $index->geo('last_login');
    $index->geoPoint('last_login');
  2. ->index($bool) field property has been replaced with ->indexField($bool); 
    // Remove from index, can't search by this field but can still use for aggregations:
       $index->integer('age')->index(false);
    $index->integer('age')->indexField(false);
  3. alias() field type has been removed. Use aliasField() instead. 
       $index->alias('comments', 'notes');
    $index->aliasField('comments', 'notes');
  4. settings() method has been replaced with withSetting() 
       $index->settings('number_of_shards', 3);
    $index->withSetting('number_of_shards', 3);
  5. map() method has been replaced with withMapping() 
       $index->map('date_detection', false);
    $index->withMapping('date_detection', false);

Schema: Blueprint analyser methods

  1. analyzer() method has been replaced with addAnalyzer() 
       $settings->analyzer('my_custom_analyzer')
    $settings->addAnalyzer('my_custom_analyzer')
  2. tokenizer() method has been replaced with addTokenizer() 
       $settings->tokenizer('punctuation')
    $settings->addTokenizer('punctuation')
  3. charFilter() method has been replaced with addCharFilter() 
       $settings->charFilter('emoticons')
    $settings->addCharFilter('emoticons')
  4. filter() method has been replaced with addFilter() 
    //Custom Field Mapping
       $settings->filter('english_stop')
    $index->addFilter('english_stop')

Dynamic Indices

  1. Dynamic indices are now managed by the DynamicIndex trait.

    namespace App\Models;
    

    use PDPhilip\OpenSearch\Eloquent\DynamicIndex;
    use PDPhilip\OpenSearch\Eloquent\Model
    

    class PageHit extends Model
    {
        use DynamicIndex;
        protected $connection = 'opensearch';
    

        protected $index = 'page_hits_*'; // Dynamic index pattern

  2. Replace setIndex() with setSuffix() in the context of saving a record.

    $pageHit = new PageHit;
    $pageHit->page_id = 4;
    $pageHit->ip = $someIp;
    // Set the specific index for the new record
     $pageHit->setIndex('page_hits_' . date('Y-m-d'));
     $pageHit->setSuffix('_' . date('Y-m-d'));
    $pageHit->save();

  3. getRecordIndex() method has been replaced with getRecordSuffix().

    $pageHit = PageHit::first();
     $index = $pageHit->getRecordIndex();
     $suffix = $pageHit->getRecordSuffix();

  4. Replace setIndex() with withSuffix() in the context of querying indexes (within the given suffix).

       $model = new PageHit;
       $model->setIndex('page_hits_2023-01-01');
       $pageHits = $model->where('page_id', 3)->get();
    $pageHits = PageHit::withSuffix('_2023-01-01')->where('page_id', 3)->get();

Medium impact changes

Result structure changes

Multiple Aggs are now returned as a flat array 

Product::agg(['avg','sum'],'orders');
v4v5
{
    "avg_orders": {
        "value": 2.5
    },
    "sum_orders": {
        "value": 5
    }
}
{
    "avg_orders": 2.5,
    "sum_orders": 5
}

Matix aggs results differ 

Product::matrix(['orders','status']);
v4v5
{
    "doc_count": 100,
    "fields": [
    {
        "name": "orders",
        "count": 100,
        "mean": 134.69999999999996,
        "variance": 5427.323232323232,
        "skewness": -0.0660090381451759,
        "kurtosis": 1.7977192917815517,
        "covariance": {
            "orders": 5427.323232323232,
            "status": -2.039393939393942
        },
        "correlation": {
            "orders": 1,
            "status": -0.010736843748959933
        }
    },
    {
        "name": "status",
        "count": 100,
        "mean": 5.169999999999999,
        "variance": 6.647575757575759,
        "skewness": -0.1108382470107292,
        "kurtosis": 1.829112534153634,
        "covariance": {
            "orders": -2.039393939393942,
            "status": 6.647575757575759
        },
        "correlation": {
            "orders": -0.010736843748959933,
            "status": 1
        }
    }
    ]
}
{
    "matrix_stats_orders": {
        "name": "orders",
        "count": 100,
        "mean": 118.60000000000001,
        "variance": 5258.121212121213,
        "skewness": -0.016424308077811454,
        "kurtosis": 1.797729283639266,
        "covariance": {
            "orders": 5258.121212121213,
            "status": -22.759595959595952
        },
        "correlation": {
            "orders": 1,
            "status": -0.12227092184902529
        }
    },
    "matrix_stats_status": {
        "name": "status",
        "count": 100,
        "mean": 4.42,
        "variance": 6.589494949494949,
        "skewness": 0.11800255330047148,
        "kurtosis": 1.806059741732609,
        "covariance": {
            "orders": -22.759595959595952,
            "status": 6.589494949494949
        },
        "correlation": {
            "orders": -0.12227092184902529,
            "status": 1
        }
    }
}

Upgrading

withoutRefresh() 

 $product->saveWithoutRefresh()
$product->withoutRefresh()->save()


 Product::createWithoutRefresh([data])
Product::withoutRefresh()->create([data])

Low impact changes

Connection

1. Database connection schema: Update as following:
'opensearch' => [
    'driver'       => 'opensearch',
    'hosts'        => explode(',', env('OS_HOSTS', 'http://localhost:9200')),
    'basic_auth'   => [
        'username' => env('OS_USERNAME', ''),
        'password' => env('OS_PASSWORD', ''),
    ],
    'sig_v4'       => [
        'provider' => env('OS_SIG_V4_PROVIDER'),
        'region'   => env('OS_SIG_V4_REGION'),
        'service'  => env('OS_SIG_V4_SERVICE'),
    ],
    'ssl'          => [
        'cert'          => env('OS_SSL_CERT', ''),
        'cert_password' => env('OS_SSL_CERT_PASSWORD', ''),
        'key'           => env('OS_SSL_KEY', ''),
        'key_password'  => env('OS_SSL_KEY_PASSWORD', ''),
    ],
    'index_prefix' => env('OS_INDEX_PREFIX', false),
    'options'      => [
        'bypass_map_validation' => env('OS_OPT_BYPASS_MAP_VALIDATION', false),
        'ssl_verification'      => env('OS_OPT_VERIFY_SSL', true),
        'retires'               => env('OS_OPT_RETRIES',null),
        'sniff_on_start'        => env('OS_OPT_SNIFF_ON_START',false),
        'logging'               => env('OS_OPT_LOGGING', false),
        'port_in_host_header'   => env('OS_OPT_PORT_HOST_HEADERS',false),
        'default_limit'         => env('OS_OPT_DEFAULT_LIMIT', 1000),
        'allow_id_sort'         => env('OS_OPT_ID_SORTABLE', false),
    ],
],

Deprecated