Search code examples
google-cloud-spanner

When a write to a table triggers a secondary index row to be created or updated, does that count as a backfill?


According to the answer here

https://stackoverflow.com/a/62524395/1060314

Read/query to the index itself are not allowed during the backfill. But writes to the original table are allowed. New writes are added to the index concurrently. After the backfill, Spanner will make sure only the latest data will be presented when queried.

So the question is, after the backfill process has completed are updates into the index as they happen treated as a similar backfill guard?

For example: if I add row Z to an existing table and that table has a secondary index that has already completed the formal backfill process where my new row Z qualifies for the index, during the time where spanner picks that write up until the time it's written to the index, is the index considered in that backfill state? Or is that backfill state only during the initial index population on an existing table?

It's possible that we write a row and then before that row actually lands in the index, another process attempts to query the index, will that query be stopped with the backfill error:

The index <index_name> cannot be used because it is in backfill


Solution

  • So the question is, after the backfill process has completed are updates into the index as they happen treated as a similar backfill guard?

    For example: if I add row Z to an existing table and that table has a secondary index that has already completed the formal backfill process where my new row Z qualifies for the index, during the time where spanner picks that write up until the time it's written to the index, is the index considered in that backfill state? Or is that backfill state only during the initial index population on an existing table?

    No, backfill only happens when an index is first created, and it only handles updates up to the timestamp of the creation. This sort of sets up a baseline for what the values should be in the index.

    New updates after this timestamp, no matter they are before or after the backfill completion, will be handled by the normal index update procedure. Note: internally Spanner does handle the update during backfill in a slightly different way to avoid contention, but that is the implementation detail.

    It's possible that we write a row and then before that row actually lands in the index, another process attempts to query the index, will that query be stopped with the backfill error:

    No. Updates to the index are committed along with the updates to the table. In your example, there should not be any error, and the reader will either see the old value if the read happens before the commit success, or the new value otherwise.