I used to fetch data by accessing the full email and using the payload/headers options, but then spotted gmail.metadata. However I have just noticed this stops us being able to use the "q" parameter to let us filter by date. My plan was the first sync did from T=0 and subsequent syncs used a "high watermark". My previously working watermark code now returns Metadata scope does not support 'q' parameter status_code: 403.
Does anyone know a way to use the GMail metadata API without having to fetch through all emails every time?
If you use the metadata scope there's no way of filtering the results you get from the query alone.
As per the documentation on users.messages: list (emphasis my own):
q:stringOnly return messages matching the specified query. Supports the same query format as the Gmail search box. For example, "
from:someuser@example.com rfc822msgid:<somemsgid@example.com> is:unread". Parameter cannot be used when accessing the api using the gmail.metadata scope.
And from the Choose Auth Scopes page:
https://www.googleapis.com/auth/gmail.metadataRead resources metadata including labels, history records, and email message headers, but not the message body or attachments.
This is by design. The q parameter allows for searching messages within fields which are restricted by the metadata scope - including message body and attachments. As a result, it is not possible whatsoever to filter results when using the metadata scope.
On the flip side however, Google already knows about this, and a Feature Request for this has been made on their Issue Tracker. You can view this feature request here, to which you can click the star (☆) in the top left to let Google know more people want this feature to be implemented.