Search code examples
androidgoogle-now

How does Google Now (Now on Tap) search inside apps?

So Google has just released a new feature for Google Now called Now on Tap. It enables a user to get contextual help from different apps (The Verge video example).

  • Does it simply take a screenshot of the current app and use OCR to extract the text?
  • Does it have "special Google permission" which allows it to read the values within the views?
Solution

  • Primarily, it should work using the assist API added to the M Developer Preview. However as "Now on Tap" is proprietary and unreleased, we can only really guess at this time.