Search code examples
c#gmail-apipayload

Unable to get payload.parts from certain mails


I'm creating a Windows service that imports mails to a application. The service is running on a Windows Server 2012 R2. The service should be able to import the mails to another application. It needs to import attachments, subject, body, senders, receivers, change labels etc. I'm currently testing the service. And it almost works well.

Problem

A small amount of mails is missing data. The api is unable to get the payload.parts when I'm requesting additional info about the mail from gmails api. Payloads.parts is null.

Tried

Since other parts of the company is dependent on mail, I have tried to forward the faulty mails to my own gmail account. So I can debug them. Unfortunately the mails don't seem to fail when they are forwarded.

I have tried to restart the service. It always fails on the same mail. Three types of the mail I've been able to debug has the mime type: "plain/text". Unsure if that makes a difference.

Edit: Found one that has mime type: "text/html"

Code

The service that syncs a messages from a label in the gmail api

 public async Task<IList<Message>> GetAllMessagesFromGoogleApi()
        {
            try
            {
                _logger.LogInformation("connecting to Gmail Api");
                var service = await _gmailMailFacade.InitializeConnectionAsync();

                _logger.LogInformation("Downloading gmail messages");
                var messages = _gmailMailFacade.GetAllMessagesWithFilter(service);
                if (messages == null)
                {
                    _logger.LogInformation("There were no messages to sync, import cancelled" + Environment.NewLine);
                    return null;
                }

                _logger.LogInformation("Downloading extra data to emails");
                var messageAdditionalData = _gmailMailFacade.GetMessageDataToMessages(messages, service);

                return messageAdditionalData;
            }
            catch (NullReferenceException e)
            {
                _logger.LogError($"Something from gmail api returned null: {e}");
                return null;
            }
            catch (Exception e)
            {
                _logger.LogError($"Unexpected exception in {nameof(GmailApiService)}. Method: {System.Reflection.MethodBase.GetCurrentMethod().Name}| e: {e.Message}");
                throw;
            }
        }

Connection to gmail runs from _gmailMailFacade.InitializeConnectionAsync()

        public readonly string[] _scopes = { GmailService.Scope.GmailModify, GmailService.Scope.GmailLabels};
        private string _applicationName = "applicationName";
        private IOptions<GmailOptions> _options;

        public GmailConnectionUtility(IOptions<GmailOptions> options)
        {
            _options = options ?? throw new ArgumentNullException(nameof(options));
        }

        public async Task<UserCredential> GetUserCredential()
        {

            var credentialPath = Path.Combine(_options.Value.CredentialBasePath, _options.Value.ClientSecretName);
            using (var stream =
                new FileStream(credentialPath, FileMode.Open, FileAccess.Read))
            {
                // The file token.json stores the user's access and refresh tokens, and is created
                // automatically when the authorization flow completes for the first time.
                string credPath = Path.Combine(_options.Value.CredentialBasePath, "token.json");
                var credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(
                    GoogleClientSecrets.Load(stream).Secrets,
                    _scopes,
                    "user",
                    CancellationToken.None,
                    new FileDataStore(credPath, true));

                Console.WriteLine("Credential file saved to: " + credPath);

                return credential;
            }
        }

        public GmailService NewGMailService(UserCredential credential)
        {
            var service = new GmailService(new BaseClientService.Initializer()
            {
                HttpClientInitializer = credential,
                ApplicationName = _applicationName,
            });


            return service;
        }

This method gets the email from gmail api. _gmailOptions contains a string with the label id. Run from _gmailMailFacade.GetAllMessagesWithFilter(service);

        public IList<Message> GetEmailsWithLabels(GmailService service)
        {
            var emailByLabel = service.Users.Messages.List("Me");
            emailByLabel.LabelIds = _gmailOptions.LabelSyncFrom;
            emailByLabel.IncludeSpamTrash = false;
            var emailsResponse = emailByLabel.Execute().Messages;
            return emailsResponse;
        }

These methods gets the payload, headers from the api

        public IList<Message> GetMailDataToMessages(IList<Message> messages, GmailService service)
        {
            return messages.Select(message => GetMailData(message, service)).ToList();
        }

        private Message GetMailData(Message message, GmailService service)
        {
            var logError = "";

            var debugMessage = message;
            try
            {
                logError = $"Unable to get extra data from gmail in {nameof(GmailApiUtility)}," +
                                 $" method: {System.Reflection.MethodBase.GetCurrentMethod().Name}," +
                                 $" on email with subject: {message?.Raw}";

                var request = service.Users.Messages.Get("me", message?.Id).Execute();

                if (request.Payload.Parts == null)
                {
                    _logger.LogError(logError);
                }

                return request;
            }
            catch (NullReferenceException e)
            {
                _logger.LogWarning($"{logError} Exception: {e}");
                return null;
            }
            catch (Exception e)
            {
                _logger.LogError($"Unexpected exception: {e}");
                throw;
            }
        }

Solution

  • For an email to have the payload.parts attribute with information, the payload mime type must be "multipart/*" for that email. That's why you're having a null payload.parts when the payload mime type is "text/plain" or "text/html".

    For example, you can send messages from Gmail UI with the plain text option activated, which will make the message payload mime type to be "text/plain". For those cases, the complete information of the message will come directly in the payload object and won't use the parts attribute.

    Here is a question where there are some examples:

    Gmail API: where to find body of email depending of mimeType