Firestore gRPC Listen does not send deletions

Question

I am using Postman to call the firestore.googleapis.com gRPC Listen. I am using the web UI (https://console.firebase.google.com/) to manually make changes to a Firestore for testing purposes. When I add a new document, I receive an appropriate document_change indicating this. However, when I delete a document, I do not receive a document_change or document_delete message at all. The count in the filter response message goes down accordingly based on how many documents I deleted, so it does appear that the deletion occurred and is recognized, but the I do not receive the change.

I am updating the resumeToken after each time I make a new request with the value from the CURRENT response for my targetId. I am deleting the documents after ending streaming and before making a new invoke and sending a new request. The procedure looks like this:

1. Invoke Listen
2. Send ListenRequest
3. Inspect responses. Get the count from the filter response for my targetId. Get the resume token from the target_change response with CURRENT for my targetId.
4. End streaming.
5. On the web UI, delete a document.
6. Invoke Listen.
7. Send ListenRequest with count and resumeToken from #3.
8. Inspect responses. Filter has a lower count for this target id. No document_change, document_delete, or document_remove received.

If relevant, I am using a bearer token to authenticate. Requests work fine and I can receive messages for new documents but not for deleted documents.

An example of my ListenRequest message:

Metadata: google-cloud-resource-prefix = projects/project1/databases/database1

{
    "database": "projects/project1/databases/database1",
    "add_target": {
        "target_id": 1,
        "resume_token": "CgkIzoO/uauyiwM=",
        "expected_count": {"value": 6},
        "query": {
            "parent": "projects/project1/databases/database1/documents",
            "structured_query": {
                "from": [
                    {
                        "collection_id": "TestCollection1",
                        "all_descendants": false
                    }
                ]
            }
        }
    },
    "once": false
}

Is my structured query incorrect? How can I write a query that includes all document changes, additions, or deletions? Or, is there another issue, such as a bug with Firestore?

Answer 1

I was able to work with Google Cloud Support in order to get a response on this issue. The resume functionality of the Firestore gRPC Listen call does not support delete capture.

In order to determine if there was a delete, you can include the expected count and then compare the last known value to the new one upon resuming. Additionally, you would need to count the number of new documents and increment your stored expected count as needed. If there is a difference between that and the new value coming from the server upon resume, that means deletes have occurred. If you wanted to know what was removed, you'd need to get all the changes again. This is not particularly helpful if your source regularly has deleted documents, but this is the intended functionality.

The resume functionality does fully support adding new documents, updating existing fields on documents, and removing fields from documents.