Search code examples
mongodbchangestream

How far back in time does a MongoDB change stream go?

I considering using MongoDB change stream feature to replicate some data from one place to another. However, I am trying to figure out of I 1) first need to do a manual one-off copy (backfill), and then 2) initialise the change stream replication - or if the change stream simply will contain all data stored in MongoDB and automatically would copy all relevant data (that is, would imply backfilling).

Solution

  • The change stream uses the oplog. Since MongoDB 4.4, it can be configured to keep the oplog for at least

    • X number of time duration.
    • Y amount of data (bytes).

    That is, one cannot use MongoDB change stream for backpropagation. Instead, backpropagation needs to happen through paginating through the actual data using the fetch API.