Apache Airflow DAG中AWS SES任务失败/重试邮件未发送问题排查

云上老周

2026-5-29

Hey there, let's tackle this Airflow + AWS SES email issue step by step. First off, the short answer to your core question: AWS SES is fully compatible with Airflow's email_on_failure and email_on_retry features—the problem is almost certainly in your configuration details, not a compatibility gap.

Let's Break Down the Issues & Fixes

1. You're Using the Wrong SES Credentials

This is the most common pitfall: Airflow's SMTP user/password for SES are not your standard IAM access keys. SES requires a dedicated set of SMTP credentials that you generate directly in the SES console. Here's how to get them:

Log into AWS SES, navigate to "SMTP Settings" in the left menu
Click "Create my SMTP credentials" to generate a unique username and password pair
Replace the smtp_user and smtp_password values in your airflow.cfg with these new credentials

2. Your SMTP Configuration Has Critical Gaps

Let's fix your airflow.cfg to align with SES requirements:

[email]
email_backend = airflow.utils.email.send_email_smtp

[smtp]
# Correct the SES SMTP endpoint (replace us-east-1 with your actual region)
smtp_host = email-smtp.us-east-1.amazonaws.com
# Use port 587 (port 25 is often blocked by cloud providers like AWS)
smtp_port = 587
smtp_starttls = True
smtp_ssl = False
# Use the SES SMTP credentials you generated earlier
smtp_user = YOUR_SES_SMTP_USERNAME
smtp_password = YOUR_SES_SMTP_PASSWORD
# Ensure this sender email is verified in AWS SES
smtp_mail_from = myemail@myjob.com

3. Task Configuration Has a Syntax Error

Look at your email parameter in the PythonOperator: you're using Chinese single quotes (’) instead of English ones ('). This will cause Airflow to misparse the email address, so fix it like this:

testfaildag_library_install_jar_jdbc = PythonOperator(
    task_id='library_install_jar',
    retries=3,
    retry_delay=timedelta(seconds=15),
    python_callable=add_library_to_cluster,
    params={'_task_id': 'cluster_create', '_cluster_name': CLUSTER_NAME, '_library_path':'s3://fakepath.jar'},
    dag=dag,
    email_on_failure=True,
    email_on_retry=True,
    email='myname@myjob.com',  # Fixed to English single quotes
    provide_context=True
)

4. SES Sandbox Mode Restrictions

If your AWS SES account is still in sandbox mode (the default for new accounts):

Both the sender (smtp_mail_from) and recipient (email) addresses must be verified in the SES console
You can't send emails to unverified addresses until you request to move out of the sandbox

Test Your Configuration Before Waiting for Task Failures

Save yourself time by testing the email setup directly via Airflow's CLI:

airflow send_email -t myname@myjob.com -s "Airflow SES Test" -m "If you get this, your SES config works!"

This will immediately tell you if your SMTP setup is correct, without waiting for a task to fail/retry.

If You Still Hit Issues

Check Airflow worker logs for SMTP-related errors (e.g., authentication failures, connection timeouts)
Verify your Airflow server's security group/firewall allows outbound traffic to port 587
Check AWS SES's "Email sending" dashboard for bounced or rejected messages

You shouldn't need to resort to on_failure_callback—getting the SMTP config right will let you use Airflow's built-in email features seamlessly with SES.

内容的提问来源于stack exchange，提问作者Zack