Search code examples
mongodbauto-increment

MongoDB auto-increment field and read order


I have a multi-instance service that writes about ~100 documents per second to a Mongo database.

Now I'm working on an application that must process those documents in the order they were inserted.

I'm stuck with this requirement of strict sequential processing order.

I'd like to avoid any queue messaging systems for this, so the easiest solution could be to add a unique auto-increment field to every document and then call the find() method to get some documents that have the field value greater than the last processed document has. I've read this official article about creating an auto-increment fields in Mongo, but does this approach guarantee the write order?

Let's say, there is this sequence of operations that happens in the order specified, but with no delay in between (with inital value of the auto-increment field = 0):

  1. Instance #1 of the service calls db.collection.insertOne() with a huge complex document #1, that may make Mongo to take some time to process it.
  2. Instance #2 calls insertOne() with a small simple document #2.
  3. My app executes find() on the same collection.

Is there any chance that the find() method in the step 3 will return only document #2 with the value of the auto-increment field = 2, and few moments later document #1 will be finally written to the database with the value = 1?

Basically, I'm afraid that the auto-increment field's value could be bumped before the corresponding document will be available for reads, so the next document could be visible to client before the first one, despite having higher field's value.

Update:

Here is a timeline of what I think could happen:

timeline


Solution

  • I found out that the situation described in my question, when the later inserted document #2 becomes visible before document #1, is not possible, at least with a standalone Mongo instance.

    MongoDB guarantees monotonic writes in this case:

    https://docs.mongodb.com/manual/core/read-isolation-consistency-recency/#monotonic-writes