Search code examples
gmail-api

Batch fetching messages performance


I need to get the last 100 messages in the INBOX (headers only). For that I'm currently using the IMAP extension to search and then fetch the messages. This is done with two requests (SEARCH and then UID FETCH).
What's the Gmail API equivalent to fetching multiple messages in one request?
All I could find is a batch API, which seems way more cumbersome (composing a long list of messages:get requests wrapped in plain HTTP code).


Solution

  • It's pretty much the same in the Gmail API as in IMAP. Two requests: first is messages.list to get the message ids. Then a (batched) message.get to retrieve the ones you want. Depending on what language you're using the client libraries may help with the batch request construction.

    A batch request is a single standard HTTP request containing multiple Google Cloud Storage JSON API calls, using the multipart/mixed content type. Within that main HTTP request, each of the parts contains a nested HTTP request.

    From: https://developers.google.com/storage/docs/json_api/v1/how-tos/batch

    It's really not that hard, took me about an hour to figure it out in python even without the python client libraries (just using httplib and mimelib).

    Here's a partial code snippet of doing it, again with direct python. Hopefully it makes it clear that's there's not too much involved:

    msg_ids = [msg['id'] for msg in body['messages']]
    headers['Content-Type'] = 'multipart/mixed; boundary=%s' % self.BOUNDARY
    
    post_body = []
    for msg_id in msg_ids:
      post_body.append(
        "--%s\n"
        "Content-Type: application/http\n\n"
        "GET /gmail/v1/users/me/messages/%s?format=raw\n"
        % (self.BOUNDARY, msg_id))
    post_body.append("--%s--\n" % self.BOUNDARY)
    post = '\n'.join(post_body)
    (headers, body) = _conn.request(
        SERVER_URL + '/batch',
        method='POST', body=post, headers=headers)