MongodbCursor , how to iterate through Huge collection?

Question

I have a MongoDb Database which has inside a collection very huge(more than 2 million of documents). I want to iterate it with a cursor. Also during the iteration I have to perform some operations with the current document.

var pending_push_cursor = collection.FindAllAs<PendingPush>();
foreach (PendingPush p_push in pending_push_cursor)
{
    operation_with(p_push)
}

The major problem is that the operation enqueues the elements, and it is desirable the iteration pause(for a few seconds) in some moments to let the operation process some elements before adding new ones.

Is there a way i can iterate the cursor in some way I can pause it , and resume later? The MongodbCursor saves the last item accessed? I only know the foreach iteration, but is there some iteration like this?

while(pending_push_cursor.isLast()){
    PendingPush p_push= pending_push_cursor.getNext()
    operation_with(p_push)
 }

If something similat exists i can save the last item queried. Thanks in advance

Answer 1

There's no problem in using a while loop with the cursor's enumerator (that's pretty much what foreach does anyway, so you can keep using that).

You should keep in mind that the cursor has a timeout after 10 minutes of inactivity which you may reach depending on your particular case. If so you can disable the timeout for that specific cursor.

Here's a simple example:

cursor.SetFlags(QueryFlags.NoCursorTimeout);
using (var enumerator = cursor.GetEnumerator())
{
    while (enumerator.MoveNext())
    {
        var item = enumerator.Current;
        // logic

        if (shouldPause)
        {
            Thread.Sleep(1000);
        }
    }
}