Infinite loop when trying to back fill existing documents Firebase extension and Firestore

Question

I'm trying to create a simple Firebase extension. I enabled the option to backfill existing users. This means that when someone installs the extension, I will add a ranking (number) in the real-time database. The problem arrives when the number of users is large and I need to perform long-running tasks. I read the docs and here is what I have tried:

exports.doBackfillExistingDocuments = tasks
    .taskQueue()
    .onDispatch(async (data) => {
      const offset = data["offset"] !== undefined ? data["offset"] : 0;

      const snapshot = await getFirestore()
          .collection("users")
          .limit(DOCS_PER_BACKFILL)
          .get();

      const processed = await Promise.allSettled(
          snapshot.docs.map(async (doc) => {
            const ref = getDatabase().ref().child("rankings").child(doc.id);
            await ref.set(0);
          }),
      );

      if (processed.length == DOCS_PER_BACKFILL) {
        const queue = getFunctions().taskQueue(
            "doBackfillExistingDocuments",
            process.env.EXT_INSTANCE_ID,
        );
        await queue.enqueue({
          offset: offset + DOCS_PER_BACKFILL,
        });
      } else {
        getExtensions().runtime()
            .setProcessingState(
                "PROCESSING_COMPLETE",
                "Backfill complete.",
            );
      }
    });

I'm a newbie in JavaScript, but per my understanding this code should get the number of documents per back fill, which is 20 and then continue until it finishes to back fill all documents. The problem arrives when I deploy the extension. The first 20 documents are added to the real-time database and then the doBackfillExistingDocuments is called over and over again, like in an infinite loop. What exactly can cause that? Please help me, because I had thousands of invocations in a very short time.

Answer 1

You could perhaps be missing the offset in your snapshot query.

const snapshot = await admin
  .firestore()
  .collection(process.env.COLLECTION_PATH)
  .offset(offset)
  .limit(DOCS_PER_BACKFILL)
  .get();

A complete example can found in the Firestore Translate extension.

source: https://github.com/firebase/extensions/blob/next/firestore-translate-text/functions/src/index.ts