Laravel: How to paginate data from Controller that is constantly being updated

Question

I have a table in the database that is constantly being updated every 1min. New rows are added, existing rows are updated, etc. Is it possible when a user first loads the page, the Controller creates a "snapshot" of the data so that when the user clicks the load more button in the blade file, it loads from that snapshot.

When the user refreshes the page, the snapshot is re-created with the updated data from the database. Not really sure where to start.. I've tried a number of things to no avail. Here is my most recent attempt:

MyController.php

public function index(Request $request)
{
    // Generate a unique snapshot ID
    $snapshotId = $request->session()->get('snapshotId', uniqid('snapshot_', true));

    // Check if this is the initial load or a subsequent "Load More" request
    $isInitialLoad = !$request->has('page');

    // On initial load, store the snapshot ID and clear previous identifiers
    if ($isInitialLoad) {
        $request->session()->put('snapshotId', $snapshotId);
        $request->session()->forget('dataIdentifiers');
    }
    
    .... get my data ....

    // Fetch identifiers (primary keys) based on the current state
    $identifiers = $dataQuery->pluck('id');

    // On initial load, store these identifiers in the session
    if ($isInitialLoad) {
        $request->session()->put("dataIdentifiers_{$snapshotId}", $identifiers->toArray());
    } else {
        // For "Load More", fetch the stored identifiers
        $identifiers = $request->session()->get("dataIdentifiers_{$snapshotId}");
    }

    // Use the identifiers to query only the items in the snapshot for pagination
    $currentPageIdentifiers = $identifiers->slice(($request->input('page', 1) - 1) * 10, 10);
    $data = Data::whereIn('id', $currentPageIdentifiers)->get();

}

And then in my blade file I have a load-more button:

    <div class="container">
        ......
    </div>

    @php
        $dataIdentifiers = session('dataIdentifiers_' . $snapshotId);
        $dataIdentifiersCount = is_array($dataIdentifiers) ? count($dataIdentifiers) : 0;
    @endphp

    @if ($dataIdentifiersCount > count((array) $data))
        <div class="text-center">
            <button id="load-more" data-next-page="{{ session('currentPage', 1) + 1 }}" data-snapshot-id="{{ $snapshotId }}" class="btn btn--theme fw-bolder px-5">Load More</button>
        </div>
    @endif

    <script>
    $(document).ready(function() {

         $('#load-more').click(function() {
            const nextPage = $(this).data('next-page');
            const snapshotId = $(this).data('snapshot-id');
            const url = `/my/controller/route?page=${nextPage}&snapshotId=${snapshotId}`;

            fetch(url)
                .then(response => response.json())
                .then(data => {
                    console.log(data);
                    $('.container').append(data.html);
                }).catch(error => console.error('Error loading more props:', error));
        });
    });
    </script>

Answer 1

One suggested solution is to cache the snapshot. Let me explain my perspective.

When the user visits your page, check for a page parameter inside the URL query. if it doesn't exist, or if its value is equal to 1, then you should refresh the snapshot in case it exists by deleting the old snapshot by its UUID using the Cache::forget() method.

Then you need to generate a new UUID to be used as an identifier to the new snapshot and cache the result of the new database query.

After that, you will retrieve the results from the cache instead of the database.

public function index(Request $request)
{
    if (! $request->has('page') || $request->integer('page') === 1) {
        $snapshot_id = session('snapshot_id');
        $key = "snapshots:$snapshot_id";
        if (Cache::has($key)) {
            Cache::forget($key);  // erase the snapshot from the cache in case it exists
        }

        $snapshot_id = Str::uuid();
        session(['snapshot_id' => $snapshot_id]); // store the snapshot id for later access

        $results = Data::all();
        Cache::forever("snapshots:$snapshot_id", $results); // cache the results inside the new snapshot
    }

    $snapshot_id = session('snapshot_id'); // retrieve the snapshot id
    $data = Cache::get("snapshots:$snapshot_id"); // get the snapshot data using the snapshot id
    $data = collect($data); // convert the data into a collection

    $per_page = 5;

    return $data->forPage($request->page, $per_page);
}