EmailOperator is not compatible with the SendGrid backend when hosted on MWAA.

[Nazarii]

Description

Modify the SendGrid to use from_email from the AIRFLOW__SMTP__SMTP_MAIL_FROM section.

Use case/motivation

When we use the EmailOperator we can not specify the email_from param.
When using the email_backend = airflow.providers.sendgrid.utils.emailer.send_email it is impossible to send the email on the MWAA since SendGrid policy requires to specify the sender email address. Sender email address can be taken from the global var SENDGRID_MAIL_FROM which is impossible to set in the MWAA. We need to use global AIRFLOW__SMTP__SMTP_MAIL_FROM section as the last chance to determine the sender email.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[uranusjr]

This sounds like something MWAA should handle instead? Airflow does not know it is being hosted on MWAA, only MWAA knows it’s hosting Airflow.

[Nazarii]

@uranusjr I don't think this is smth to fixed on the MWAA, sengrid provider should follow the similar flow defined in EmailOperator:

1. When sending emails with EmailOperator we do not specify the sender email address. This address will be taken from the AIRFLOW__SMTP__SMTP_MAIL_FROM setting.
2. Sending an email with the SendGrid provider as a backend requires setting up the SENDGRID_MAIL_FROM. It will be good if we have a single entry point configuration for different mail systems, so we do not set up the sender email per provider. What I propose is to use AIRFLOW__SMTP__SMTP_MAIL_FROM option if SENDGRID_MAIL_FROM is not set.

[uranusjr]

If the problem is speficically about using something other than SENDGRID_MAIL_FROM, 2.3.0 will introduce a AIRFLOW__EMAIL__FROM_EMAIL (#18042) that covers this usage. Note that the SMTP configuration is not used because, well, Sendgrid is not SMTP.

Closing as the usage is already fixed from Airflow’s perspective. Whether AIRFLOW__EMAIL__FROM_EMAIL will be available on MWAA when they make 2.3.0 available is only relevant to MWAA.

[Nazarii]

Looks good, AIRFLOW__EMAIL__FROM_EMAIL will cover this. Thank you.