Search code examples
google-cloud-platformgmailgoogle-cloud-pubsub

Google Cloud Pub/Sub to watch Gmail


Introduction
I am looking to use the Pub/Sub API to monitor the inbox of an account in our organization. It is just a normal Gmail inbox, but it receives IT related notifications important to our business. Some of the emails are not that high of priority and can be checked by ourselves when the time comes, however other emails of high importance(such as our ISP performing maintenance) we would like to notify corporate team members in the office of the events that are happening.

Example
For example, our ISP alerts us(via email) that there is going to be Maintenance Work done on July 9th, 2022 from 12 AM - 6 AM, I would like to forward that email to our Warehouse Manager to make him aware of the maintenance that is happening.

Question
All of the documentation and other posts that I have found are typically using a database to hold the messages their "publisher" is pushing and are using their own application as the "subscriber" to the events being pushed. Do I need to create a database and application to perform the functionality that I desire? Am I overthinking this and making it more complicated than it needs to be and only one or so API call needs to be made


Solution

  • If you just need to forward the email to a known address and you already know the sender's (ISP) email as well as the subject and/or keywords, you don't need to develop an application at all. You can just go into your inbox settings, create a filter, set the From address and Subject and finally set the forwarding address to your warehouse manager's email address.


    If you're doing something more complicated like notifying people via a different channel or you have some custom logic to detect whether the email's contents are important to you or not then you'll have to write an application.

    Normally you'd create a Watch request on the target inbox by using the Gmail API. Once you do that you will receive a historyId which corresponds to the current state of your inbox. You should store this id. Your application will also begin to receive notifications via Pub/Sub whenever there's a new change to your inbox. These notifications also contain history ids.

    The benefit of having a database to store history ids is that every time you receive a new notification, you can pull the last known id from your DB and call the List History method/endpoint setting the startHistoryId as the id you just got from your DB. That will make sure you only retrieve the changes that have happened to your inbox since the last notification you processed. In other words you're only going to be processing new messages, which should be a smaller changeset than if you were to list all the messages every time. Without storing some kind of state you're going to be re-scanning messages that you've already seen before. Once you've processed all the new changes in the diff you can update the DB with the new history id.

    Technically you don't need to do all of that. You could periodically poll the inbox calling List Messages every time but then you'd need to make other considerations like how often to poll and how many messages you want to list each time.