Search code examples
pythongoogle-app-enginegoogle-cloud-datastoregoogle-search-api

Integrating GAE Search API with Datatstore


When a document is stored into both the Cloud datastore and a Search index, is it possible when querying from the Search index, rather than returning the Search index documents, returning each corresponding entity from the Cloud datastore instead? In other words, I essentially want my search query to return what a datastore query would return.

More background: When I create an entity in the datastore, I pass the entity id, name, and description parameters. A search document is built so that its doc id is the same as the corresponding entity id. The goal is to create a front-end search implementation that will utilize the full-text search api to retrieve all relevant documents based on the text query. However, I want to return all details of that document, which is stored in the datastore entity.

Would the only way to do this be to create a key for each search doc_id returned from the query, and then use get_multi(keys) to retrieve all relevant datastore entities?


Solution

  • There is no first class support for this, your best bet is to make the document id match the datastore key and route all put/get/search requests through a single DAO/repository tier to ensure some level of consistency.

    You can use parallel Async writes to keep latency down, but there's not much you can do about search not participating in transactions. It also has no defined consistency, so assume it is eventual, and probably much slower than datastore index propagation.