Firestore Pagination: Updated realtime data moving between pages unexpectedly

Question

I'm using Firestore and querying data with orderBy set to updatedAt in descending order, and limiting the results to 2 items per page (for testing purpose). However, I'm encountering unexpected behavior when updating data.

When I update the oldest item on the second page to have the latest updatedAt timestamp, it moves to the first position on the first page, effectively removing it from the second page. Additionally, the second item on the first page also gets removed.

Here's a breakdown of the situation:

Initial state:

Page 1: ['item1', 'item2'] (no `startAfter` specified)
Page 2: ['item3', 'item4'] (starting after 'item2')

After updating item4 (updatedAt changed to latest):

Page 1: ['item4', 'item1']
Page 2: ['item3']

However, my expected outcome is:

Page 1: ['item4', 'item1']
Page 2: ['item2', 'item3']

// Query example
const query = firestore.collection('myCollection')
                     .where('tests', 'array-contains', `test`)
                     .orderBy('updatedAt', 'desc')
                     .limit(2)
                     .startAfter(startAfterVar)
                     .onSnapshot(snapshot => { 
                       // Do something to store
                     });

Your assistance in resolving this issue is greatly appreciated. Thank you!

Updated 1:

Q: So you basically say that item2 is deleted from the database? Or isn't just displayed?

A: Basically, item2 isn't displayed, not deleted from DB.

Q: Can you please edit your question and add your database structure as a screenshot?

A: Data Structure:

myCollection (collection)
  |
  --- item1 (document)
       |
       --- tests: [array string]
       --- updatedAt: [timestamp]
       --- otherFields: ...

Updated 2:

I'm implementing infinite pagination with my query

Update 3:

Updated screenshot of Topics collection

My actual query is

// Query example
const query = firestore.collection('topics')
                     .where('members', 'array-contains', userID)
                     .orderBy('lastLoggedAt', 'desc')
                     .limit(2)
                     .startAfter(startAfterVar)
                     .onSnapshot(snapshot => { 
                       // Do something to store
                     });

Answer 1

Disclaimer: pagination with realtime listeners is hard.

What you're describing is the expected behavior. Starting with this data set:

[item1, item2, item3, item4]

When you set up the first listener, you read the two documents for item1 and item2. Then you create the second listener to start after item2, and that then reads item3 and item4.

Now you change item4 to become the first one in the sort order. So it becomes:

[item4, item1, item2, item3]

So the first listener now sees item4 and item1. But the second listener still starts after item2, so it only sees item3 and you effectively hide item2.

---

Since moving an item changes the cursor of all pages after it, you will need to reanchor all listeners to their new cursor documents. So you will need to detach all listeners and recreate them, or create additional listeners for the documents that are falling in between the existing listeners.

This is precisely why the FirebaseUI library only supports pagination without realtime listeners: pagination with realtime data updates is hard.