A Guide to Laravel Lazy Collection

The LazyCollection is the class that utilizes PHP generators to augment the robust Collection class. This enables you to keep your memory usage to a minimum when working with large sets of data. 

Let’s say your app is requested to process a log file of a few gigabytes and use collection methods from Laravel to parse the logs. Using Lazy Collections allows you to keep a small part of file memory at any given time instead of reading the entire file in the memory all at once. 

1. What is the Laravel Lazy Collection?

A collection in Laravel keeps postponing the processing of the data until its actual need arises. This collection is called a Lazy Collection. This trick comes in handy while working with large datasets. It also makes the data processing more memory efficient. 

Call the LazyCollection::make method and provide a callback function that returns the iterator. This creates a Lazy Collection. When you need the data, execute the callback function. It allows the iterator to retrieve the data from the callback function. The use of Lazy Collection for processing large sets of data is shown in the example below:

use Illuminate\Support\LazyCollection;
 
$users = LazyCollection::make(function () {
    $handle = fopen('users.csv', 'r');
 
    while ($row = fgetcsv($handle)) {
        yield $row;
    }
});
 
$maleUsers = $users->filter(function ($user) {
    return isset($user[2]) && strtolower($user[2]) === ‘male’;
})->each(function ($user) {
    // Process the male user
    print_r($user);
});

Using the LazyCollection::make method, we created a Lazy Collection in the above example that can read the data from the users.csv file one row at a time. A filter method is used to filter the collection to allow it to include only users whose gender is identified as male. In the filtered collection, each user is processed using this method. 

Loading the whole dataset in the memory at once is not possible because Lazy Collection doesn’t process the data until the actual need arises. This approach is more memory-efficient in comparison to loading the whole dataset in a regular collection before filtering and processing it. 

2. How to Create a Lazy Collection in Laravel?

Just pass the PHP generator function to the make method of the collection to build a Lazy Collection instance

use Illuminate\Support\LazyCollection;
 
LazyCollection::make(function () {
   $handle = fopen('log.txt', 'r');
 
   while (($line = fgets($handle)) !== false) {
       yield $line;
   }
});

3. How to Convert Regular Connection into a Lazy Collection

Let’s understand the process by taking an example of a large dataset like the results of an API call that can’t be streamed.

use Illuminate\Support\Collection;
 
function get_all_customers_from_quickbooks() : Collection
{
    // Run some code that gets all QuickBooks customers,
    // and return it as a regular, eager collection...
}

We do not care about the function’s implementation. The actual importance lies in the huge collection that returns to us that we can’t stream but needs to be stored in the memory. Here, the LazyCollection becomes helpful. For example, it helps us count the number of customers in France who have more than 100 euros in balance.  

$count = get_all_customers_from_quickbooks()
    ->where('country', 'FR')
    ->whereBetween('balance', [100, 999])
    ->count();

Although it looks straightforward, it isn’t. Every time we call, we create a new collection which generates another array in the memory to store all the new filtered values. We can improve this process. 

It doesn’t matter if all the original values are stored in the memory, it’s the subsequent filters that shouldn’t be there. Using the lazy method, we can convert the regular Collection class into a LazyCollection.

$count = get_all_customers_from_quickbooks()
    ->lazy()
    ->where('country', 'FR')
    ->whereBetween('balance', [100, 999])
    ->count();

This piece of code helps you keep all the original values in the memory and no additional memory is allocated upon filtering of the results. 

4. Usage of Lazy Collection Class

Lazy collection class is mainly designed to keep memory usage low by the Laravel applications. It is using the power of PHP Generators and allows us to work with large databases while using low memory usage.

Let’s assume that your application database table has a lot of data and you want to retrieve those records for your website requirement OR exporting to excel OR any file.

Imagine that your application is using all() method of Laravel Eloquent then you may face out of memory error and it is happening because we are retrieving all records in one go and it will store into memory so memory usage will be high at that time. If you are doing so then your application will be down for some time.

Laravel 6 comes up with some extremely strong features which allow website users to run smoothly by using low memory.

5. Retrieving Records using Cursor Method

Laravel Lazy collection cursor method that allows us to retrieve no of records from the given models and then it will return a Lazy Collection object. If you look at the below example then you can find that we are effortlessly generating all the models from the Lazy collection object.

$products = \App\Product::cursor();
 
foreach($products as $key => $product) {
    echo "Product Name: ". $product->name ."<br>";
    echo "Description: ". $product->description ."<br>";
    echo "Price:". $product->price ."<br>";
    echo "<br>";
}

In the above example, SQL query is only going to be executed when each loop will start executing. Below code is reference to the cursor method definition in which yield is used for generators (and will return a Lazy collection object which is iterable using loops).

Please find the cursor method definition in the code below.

public function cursor()
{
    if (is_null($this->columns)) {
        $this->columns = ['*'];
    }
 
    return new LazyCollection(function () {
        yield from $this->connection->cursor(
            $this->toSql(), $this->getBindings(), ! $this->useWritePdo
        );
    });
}

6. How to Filter Records using Lazy Collection:

Lazy collection Cursor method provides facility to filter records and this one of the many advantages comes along with Laravel Framework . For example, if you want some specific records from the thousands of the rows then you can filter records by using the filter method of the cursor.

Here you can find example of filtering records using Lazy collection class:

$products = \App\Product::cursor()->filter(function ($product) {
	return $product->price >= 5000 && $product->price <= 10000;
});
 
foreach($products as $key => $product) {
    echo "Product Name: ". $product->name ."<br>";
    echo "Description: ". $product->description ."<br>";
    echo "Price:". $product->price ."<br>";
    echo "<br>";
}

In the above example, you can see that We have filter records from 5000 so that means We will get rows after 5000 rows.

7. How to read log file using Lazy Collection Class:

If you want to read any file OR let’s say a log file with the help of Lazy collection class then you can refer below code. Please note that we are reading log files with minimum usage of system memory.

Let’s take an example of 60,000 lines of log file. First we will need to initialize a lazy collection class to use make methods of that class.

use Illuminate\Support\LazyCollection;
 
$filelogs = LazyCollection::make(function () {
    $filehandle = fopen('./logfile.txt', 'r');
    while (($fileline = fgets($filehandle)) !== false) {
        yield $fileline;
    }
});
 
foreach ($filelogs as $fileline) {
   echo $fileline . '<br>';
}

If you look at the above example then you can see we have called a lazy collection class to read a log file to read each line and yield it to print that line. Our goal to use a lazy collection class is to read big log file without application down. If we use array instead of yield then the application will go down while reading a large log file.

8. Lazy Collection Methods

Lets delve deep into the Lazy collection methods.

8.1 tapEach()

Lazy collection class can access all the methods of collection class and additionally lazy collection class has tapEach() method. It is a similar method from Collection class but with a useful improvement.

In the default collection class, if you are using a callback function then it will call each element from the collection object immediately but tapEach() method will call the element when we are trying to ask the callback function to pull that element from the given list so normally it operates element one by one.

use Illuminate\Support\LazyCollection;
Route::get('/', function () {
    $lazyCollection = LazyCollection::times(INF)->tapEach(function ($value) {
        dump($value);
    });
$array = $lazyCollection->take(250)->all();
dump($array);
 
});

Here you can see that we are calling taking 250 from the list so it means that if we comment line $array = $lazyCollection-> take(250)->all();  then nothing will be executed or dumped yet. Lazy collection class takes records from the tables when we call the method to execute otherwise it will return empty.

8.2 takeUntilTimeout()

The values are enumerated by the new lazy collection that is returned by the takeUntilTimeout method. The collection then stops the enumeration.

$lazyCollection = LazyCollection::times(INF)
   ->takeUntilTimeout(now()->addMinute());
 
$lazyCollection->each(function (int $number) {
   dump($number);
 
   sleep(1);
});
 
// 1
// 2
// ...
// 58
// 59

To provide a use case for this method, let’s take an example of an app that uses a cursor to submit invoices from the database. The below code will help you define scheduled tasks that run every 10 minutes and the invoices should be processed for no more than 9 minutes of timeframe. 

use App\Models\Invoice;
use Illuminate\Support\Carbon;
 
Invoice::pending()->cursor()
   ->takeUntilTimeout(
       Carbon::createFromTimestamp(LARAVEL_START)->add(9, 'minutes')
   )
   ->each(fn (Invoice $invoice) => $invoice->submit());

8.3 remember()

To remember the values that are already enumerated, the remember method is used which returns a lazy collection. The remembered values are not retrieved again on the next collection enumerations. 

// No query has been executed yet...
$users = User::cursor()->remember();
 
// The query is executed...
// The first 5 users are hydrated from the database...
$users->take(5)->all();
 
// First 5 users come from the collection's cache...
// The rest are hydrated from the database...
$users->take(20)->all();

9. Conclusion

We’re pretty much sure that the Lazy collection class is going to be used by many Laravel applications and We hope this blog will surely help to get knowledge and speed up your application performance. Feel free to contact us regarding Laravel development.

FAQs:

What is lazy load in Laravel?

In Laravel, lazy loading is the dynamic property that will load their relationship data only when you actually access them. 

How to prevent lazy loading in Laravel?

You can prevent lazy loading in Laravel by using the preventLazyLoading method on the Model class. It completely disables the lazy loading.

What is a lazy collection?

The lazy collection is similar to the Base collection as both have the same elements. The only difference is that in a lazy collection, operations like filter and map are executed lazily.