Migrations

OpenSearch index management differs significantly from traditional SQL schema operations. This package provides a fully custom schema layer tailored to OpenSearch’s structure and capabilities.

Migration Class Creation

To use migrations for index management, you can create a standard Laravel migration class. However, it’s crucial to note that the up() and down() methods encapsulate the specific OpenSearch-related operations, namely:

Schema Management: Utilizes the PDPhilip\OpenSearch\Schema\Schema class.
Index/Analyzer Blueprint Definition: Leverages the PDPhilip\OpenSearch\Schema\Blueprint class for defining index & analyser structures.

Full example

<?php

use Illuminate\Database\Migrations\Migration;
use PDPhilip\OpenSearch\Schema\Schema;
use PDPhilip\OpenSearch\Schema\Blueprint;

class MyIndexes extends Migration
{
    public function up()
    {
        Schema::create('contacts', function (Blueprint $index) {
            //first_name & last_name are automatically added to this field
            //you can search by full_name without ever writing to full_name
            $index->text('first_name')->copyTo('full_name');
            $index->text('last_name')->copyTo('full_name');
            $index->text('full_name');

            //Multiple types => Order matters ::
            // Top-level `email` will be indexed as a text field
            // Subfield `email.keyword` can be used for sorting in queries (e.g., ->orderBy('email.keyword'))
            $index->text('email');
            $index->keyword('email');

            //Dates have an optional formatting as second parameter
            $index->date('first_contact', 'epoch_second');

            //Nested properties (optional properties callback to define nested properties)
            $index->nested('comments')->properties(function (Blueprint $nested) {
                $nested->keyword('name');
                $nested->text('comment');
                $nested->keyword('country');
                $nested->integer('likes');
                $nested->date('created_at');
            });

            // To define object fields without full nesting, use dot notation.
            $index->text('products.name');
            $index->float('products.price')->coerce(false);

            //Disk space considerations ::
            //Not indexed and not searchable:
            $index->keyword('internal_notes')->docValues(false);
            //Remove scoring for search:
            $index->text('tags')->norms(false);
            //Remove from index, can't search by this field but can still use for aggregations:
            $index->integer('score')->indexField(false);

            //If null is passed as value, then it will be saved as 'NA' which is searchable
            $index->keyword('favorite_color')->nullValue('NA');

            //Numeric Types
            $index->integer('some_int');
            $index->float('some_float');
            $index->double('some_double');
            $index->long('some_long');
            $index->short('some_short');
            $index->byte('some_byte');
            $index->halfFloat('some_half_float');
            $index->scaledFloat('some_scaled_float',140);
            $index->unsignedLong('some_unsigned_long');

            //Alias Example
            $index->text('notes');
            $index->aliasField('comments', 'notes');

            $index->geoPoint('last_login');
            $index->date('created_at');
            $index->date('updated_at');

            //Settings
            $index->withSetting('number_of_shards', 3);
            $index->withSetting('number_of_replicas', 2);

            //Other Mappings
            $index->withMapping('dynamic', false);
            $index->withMapping('date_detection', false);

            //Custom Mapping
            $index->property('flattened','purchase_history');


            //Custom Analyzer Setup:

            //Analyzer Setup
            $index->addAnalyzer('my_custom_analyzer')
                ->type('custom')
                ->tokenizer('punctuation')
                ->filter(['lowercase', 'english_stop'])
                ->charFilter(['emoticons']);
            //Tokenizer Setup
            $index->addTokenizer('punctuation')
                ->type('pattern')
                ->pattern('[ .,!?]');
            //CharFilter Setup
            $index->addCharFilter('emoticons')
                ->type('mapping')
                ->mappings([":) => _happy_", ":( => _sad_"]);
            //Filter Setup
            $index->addFilter('english_stop')
                ->type('stop')
                ->stopwords('_english_');
            //Normalizer Setup
            $index->addNormalizer('my_normalizer')
                ->type('custom')
                ->charFilter([])
                ->filter(['lowercase', 'asciifolding']);
        });
    }

    public function down()
    {
        Schema::deleteIfExists('contacts');
    }
}

Example:

$index->property('date', 'last_seen', [
    'format' => 'epoch_second||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd',
    'ignore_malformed' => true,
]);

Index Creation

Schema::create

Creates a new index with the specified structure and settings.

Schema::create('my_index', function (Blueprint $index) {
    // Define fields, settings, and mappings
});

Schema::createIfNotExists

Creates a new index only if it does not already exist.

Schema::createIfNotExists('my_index', function (Blueprint $index) {
    // Define fields, settings, and mappings
});

Index Deletion

Schema::delete

Deletes the specified index. Will throw an exception if the index does not exist.

// Boolean
Schema::delete('my_index');

Schema::deleteIfExists

Deletes the specified index if it exists.

// Boolean
Schema::deleteIfExists('my_index');

Index Lookup and Information Retrieval

Schema::getIndex

Retrieves detailed information about a specific index or indices matching a pattern.

Schema::getIndex('my_index');
Schema::getIndex('page_hits_*');
//or
Schema::getTable('my_index');

Schema::getIndices

Equivalent to Schema::getIndex('*'), retrieves full information about all indices on the opensearch cluster.

Schema::getIndices();

Schema::getIndicesSummary

Alternative Schema::getTables()

Retrieves information about all indices on the opensearch cluster.

Schema::getIndicesSummary();

...
{
    "name": "user_logs",
    "status": "open",
    "health": "yellow",
    "uuid": "UfLaQLcWSHSP_rVaWZuibA",
    "docs_count": "12554",
    "docs_deleted": "0",
    "store_size": "2.3mb"
},
...

Schema::getMappings

Retrieves the mappings for a specified index.

Schema::getMappings('my_index');

{
    "color": {
        "type": "text"
    },
    "color.keyword": {
        "type": "keyword",
        "ignore_above": 256
    },
    "created_at": {
        "type": "date"
    },
    "description": {
        "type": "text"
    },
    "description.keyword": {
        "type": "keyword",
        "ignore_above": 256
    },
    "is_active": {
        "type": "boolean"
    },
    "last_order_datetime": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
    },
    "last_order_ts": {
        "type": "date",
        "format": "epoch_millis||epoch_second"
    },
    "manufacturer": [],
    "manufacturer.country": {
        "type": "text"
    },
    "manufacturer.country.keyword": {
        "type": "keyword",
        "ignore_above": 256
    },
    "manufacturer.location": {
        "type": "geo_point"
    },
    "manufacturer.name": {
        "type": "text"
    },
    "manufacturer.name.keyword": {
        "type": "keyword",
        "ignore_above": 256
    },
    "name": {
        "type": "text"
    },
    "name.keyword": {
        "type": "keyword",
        "ignore_above": 256
    },
    "updated_at": {
        "type": "date"
    }
}

Raw mapping

Schema::getMappings('my_index',true);

{
    "my_index": {
        "mappings": {
            "properties": {
                "color": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "created_at": {
                    "type": "date"
                },
                "description": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "is_active": {
                    "type": "boolean"
                },
                "last_order_datetime": {
                    "type": "date",
                    "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
                },
                "last_order_ts": {
                    "type": "date",
                    "format": "epoch_millis||epoch_second"
                },
                "manufacturer": {
                    "properties": {
                        "country": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        },
                        "location": {
                            "type": "geo_point"
                        },
                        "name": {
                            "type": "text",
                            "fields": {
                                "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                }
                            }
                        }
                    }
                },
                "name": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "updated_at": {
                    "type": "date"
                }
            }
        }
    }
}

Schema::getSettings

Retrieves the settings for a specified index.

Schema::getSettings('my_index');

{
    "my_index": {
        "settings": {
            "index": {
                "routing": {
                    "allocation": {
                        "include": {
                            "_tier_preference": "data_content"
                        }
                    }
                },
                "number_of_shards": "1",
                "provided_name": "my_index",
                "creation_date": "1742830787691",
                "number_of_replicas": "1",
                "uuid": "g1Jzm6_ORA6dGftm59asKQ",
                "version": {
                    "created": "8521000"
                }
            }
        }
    }
}

Schema::hasField

Checks if a specific field exists in the index’s mappings.

// Boolean
Schema::hasField('my_index', 'my_field');

Schema::hasFields

Checks if multiple fields exist in the index’s mappings.

// Boolean, true if all fields exist
Schema::hasFields('my_index', ['field1', 'field2']);

Schema::indexExists

Checks if a specific index exists.

// Boolean
Schema::indexExists('my_index');

Schema::getFieldMapping

Returns the mapping for a specific field

Schema method that can be called from your model:

Product::getFieldMapping('color'); //Returns a key/value array of field/types for color
Product::getFieldMapping('color',true); //Returns the mapping for color field as is from opensearch
Product::getFieldMapping(['color','name']); //Returns mappings for color and name
Product::getFieldMapping(); //returns all field mappings, same as getFieldMapping('*')

Product::getFieldMapping();

{
    "color": "text",
    "color.keyword": "keyword",
    "created_at": "date",
    "datetime": "text",
    "datetime.keyword": "keyword",
    "description": "text",
    "description.keyword": "keyword",
    "is_active": "boolean",
    "last_order_datetime": "date",
    "last_order_ts": "date",
    "manufacturer.country": "text",
    "manufacturer.country.keyword": "keyword",
    "manufacturer.location": "geo_point",
    "manufacturer.name": "text",
    "manufacturer.name.keyword": "keyword",
    "name": "text",
    "name.keyword": "keyword",
    "updated_at": "date"
}

or via Schema: Schema::getFieldMapping($index, $field, $raw)

Schema::getFieldMapping('products','color',true);

Prefix Management

Schema::overridePrefix

Temporarily overrides the default index prefix for subsequent operations. This can be useful in multi-tenant applications or when accessing indices across different environments.

Schema::overridePrefix('some_other_prefix')->getIndex('my_index');

Direct DSL Access

Provides direct access to the OpenSearch DSL, allowing for custom queries and operations not covered by other methods.

Schema::indices()->close(['index' => 'my_index']);