Search code examples
c#google-cloud-datastore

Is there a way to update indexes.yaml file of Google Cloud Datastore using C#

I'm working on a project that uses Datastore to keep data of in-game events for machine learning. I hope I'm able to update the indexes.yaml file on production mode by API calling or some other methods, instead of running gcloud SDK commands manualy. Is there any way to reach this purpose?

[Architecture]

  • The language we're using is C# .Net core 2.2
  • We're using "GAME IDs" as the Kinds in Datastore. Each time we have a new game go online, there will be a new Kind created (by C# code).
  • Each Kind keeps similar properties, but not all the same.
  • All kinds use a common set of properties as the composite index (let's say: UserId, TimeStamp and EventType)

[My Current Approach]

  • I'm trying to apply the # AUTOGENERATED function into my Datastore, but it only works on my emulator Datastore
  • So far I'm able to append a new composite index into the indexes.yaml file of my emulator

  • But the indexes are not updating on the production Datastore, I have to append the indexes on emulator then update it to production by

    gcloud datastore indexes create

    command (though it makes sense as the official documents only mentioned this approach)

[What I need]

Because we may have many new games go online in the coming months, so it will be hard to add then update indexes for these new games one by one. Is there any way to automatically update the indexes.yaml on production Datastore?

My appreciates for giving any suggestion or reference resources!

Solution

  • Your current approach only works in DataStore emulator. It is intended behavior. So you can identify your indexes and then deploy them to production database.

    Cloud Firestore in Datastore and Firestore native modes creates automatically only single indexes and not composite indexes.

    However, using Firestore (Datastore in Firestore mode) it is possible to identify what indexes you are lacking using error message.

    As a workaround (also for Firestore mode) you can:

    1. Create composite indexes using API call, or
    2. Create composite indexes using client libraries