Chunking

In scenarios where you're dealing with large datasets, Laravel's chunk method becomes invaluable, allowing you to process large sets of results in manageable portions. This approach is particularly useful in memory-intensive operations, such as batch updates or exports, where loading the entire dataset into memory at once could lead to performance issues. The Laravel-OpenSearch integration adapts this familiar concept from Laravel Eloquent ORM to work within the context of OpenSearch data.

Basic Chunking

The chunk method breaks down the query results into smaller "chunks" of a specified size, executing a callback function for each chunk. This allows you to operate on a subset of the results at a time, reducing memory usage.

Product::chunk(1000, function ($products) {
    foreach ($products as $product) {
        // Example operation: Increase price by 10%
        $product->price *= 1.1;
        $product->save();
    }
});

Chunk By Id

chunkById($count, callable $callback, $column = '_id', $alias = null, $keepAlive = '5m')

The chunkById() method is a specialized version of the chunk method that paginates results based on a unique identifier ensuring that each chunk contains unique records.

Any ordering clauses will be ignored as this method uses the unique identifier to paginate the results.

If no identifier is provided (default value _id), the chunk will use PIT (point in time).

keepAlive parameter uses OpenSearch's time units, such as 1m for 1 minute, 1h for 1 hour, and 1d for 1 day.

Product::chunkById(1000, function ($products) {
    foreach ($products as $product) {
        // Example operation: Increase price by 10%
        $product->price *= 1.1;
        $product->save();
    }
}, 'product_sku.keyword');