Google SMTP returns required authentication but it's done as an app password

Question

I am setting up SMTP service on Airflow 2.1.2. I get this error which is quite obvious:

smtplib.SMTPSenderRefused: (530, b'5.7.0 Authentication Required. Learn more at\n5.7.0  https://support.google.com/mail/?p=WantAuthError w5sm3433302wra.87 - gsmtp'

But authentication is done, i am using an email that has 2 steps authentication, so I created an app password and I recognized the activity in google account security tab..

My setup is:

AIRFLOW__EMAIL__EMAIL_BACKEND   airflow.utils.email.send_email_smtp
AIRFLOW__SMTP__SMTP_HOST    smtp.gmail.com
AIRFLOW__SMTP__SMTP_MAIL_FROM   my email
AIRFLOW__SMTP__SMTP_PASSWORD    my app pwd of 16 chars
AIRFLOW__SMTP__SMTP_PORT    587
AIRFLOW__SMTP__SMTP_SSL False
AIRFLOW__SMTP__SMTP_STARTTLS    True
AIRFLOW__SMTP__SMTP_USER    my email

I checked and there is no extra space or something like that neither in pwd or email.

Why is it returning required authentication when authentication is really done?

Answer 1

Not sure why it is not working. But do you see:

Fetching SMTP credentials from configuration variables will be deprecated in a future warning? You should if your environment variables will be used.

If you do not see it, then you likely have smtp_default connection in the Connection database.

In Airflow 2 the authentication for SMTP has been brought to the same level as other connections and rather than passing user/password via environment variable, they should be configured as login/password in the 'smtp_default' connection.

The variables should still work as fallback, but if you already have smtp_default connection created and has some login/password (even empty) configured, those will be used instead of the ENV vars.

In your case likely the solution will be to set login/password in the connection.