Search code examples
emailsmtpgmailsmtplib

Issues sending email through Google's SMTP Relay

My Ubuntu based webserver needs to occasionally send emails. My python code is:

withsmtplib.SMTP('smtp-relay.gmail.com', 587, 'mydomain.com') as s:
    s.sendmail(fromaddr, toaddr, msg.as_string())
    s.quit()

I have

  • a Google workspace account
  • am using IP authentication (not SMTP auth)
  • my staging and production servers added as trusted IPs (staging is local, production is cloud)

This setup had been working fine for 6+ months.

Two days ago I upgraded Ubuntu from 20LTS to 22LTS and python 3.8 to 3.10. Now the email is working fine on the staging server, but production keeps throwing:

Invalid credentials for relay [...]. The IP\n5.7.1 address you've registered in your G Suite SMTP Relay 
service doesn't\n5.7.7 match domain of the account this email is being sent from. If you are\n5.7.1 trying to 
relay mail from a domain that isn't registered under your G\n5.7.1 Suite account or has empty envelope-from, 
you must configure your\n5.7.1 mail server either to use SMTP AUTH to identify the sending domain or\n5.7.1 to 
present one of your domain names in the HELO or EHLO command. For\n5.7.1 more information, please visit 
https://support.google.com/a/answer/6140680#invalidcred ...

Any suggestions?

Edit 1: I fired up my old ubuntu server in the cloud. I added its new IP as trusted on Google. The email worked fine. I can think of only three possibilities

  • Google somehow recognizes and trusts requests coming from the old device (even though it now has a different IP)
  • Linode is somehow not sending the correct IP address from my new server
  • Something broke during the Ubuntu upgrade

I find each of the 3 possibilities quite bizarre and unbelievable at this point, but I'll keep researching.

PS: Three factoids that may/may not be relevant:

  • I upgraded the staging server in place. For production I spun a new instance, made sure everything else was working fine (except email) and then transferred IP from the existing instance to new
  • When I log in to my google admin account to edit trusted IP list, my IP is the same as staging server. I don't think I have the same option for production, since it's an Ubuntu server I manage through SSH
  • I found some comments online (none in official documentation), that the reverse DNS needs to be setup before Google would relay anything. I set up the entry about 20 hours ago for production, but still getting the same error. And for my staging server, I don't have rDNS and it still sends emails (it's accessible from the internet, but I don't have a static IP)

PPS:

  • The sender email is someuser@mydomain.com (not @gmail.com)
  • The production server is hosted on linode.com
  • This post comes close to discussing a similar situation, but that is focused more on signing in. My setup uses IP authentication, not SMTP auth. Plus it was working fine until Friday (8/12)
Solution

  • It turned out to be a really frustrating issue. My best guess is that Linode's Ubuntu 22.04 repository has issues. We were thinking of migrating to AWS anyway, this gave us a strong impetus.

    Anyway, here are some tips from my experience that a future reader might be able to benefit from

    1. When you're using IP authentication for Google's SMTP relay, the updates are fairly quick. I ended up spinning at least 5 instances with 5 different IPs, and each time Google was able to trust my IP within 2-3 seconds (after I updated in workspace console)
    2. Google didn't care about my reverse DNS entries. I had read some comments online that Google wouldn't relay without rDNS, but I didn't face any such problems (at least not any rDNS I was setting. The ISP or the cloud provider have a default entry, that was good enough - if Google was even looking for it). This one was particularly problematic because that information can take hours to propagate, and I kept thinking maybe my code will start to work tomorrow.
    1. The error message I received from Google was pretty uninformative. I contacted Google support to see if they have access to anything more meaningful on the server side. They didn't - it was a waste of time
    2. It was somewhat helpful to run a fake SMTP server to see what my client was sending. I got it from this post. I ran it for a setup that was working and one that wasn't. In my case, the communication received was identical. Though in hindsight maybe I would've seen some differences if I ran it on a remote server.
    python -m smtpd -n -d -c DebuggingServer localhost:2500