Airflow DAG中SimpleHttpOperator运行时Basic Auth配置问题

错误原因 #

你遇到的TypeError是因为auth_type参数要求传入**requests.auth.AuthBase的子类（类本身）**，而不是已经实例化的对象。当你传入HTTPBasicAuth('user','pass')实例时，Airflow会尝试调用这个实例作为认证函数，导致缺少必需的参数r。

解决方法 #

方法1：使用auth_type+auth_args参数（推荐） #

直接传入HTTPBasicAuth类，并用auth_args传递用户名和密码，Airflow会自动帮你实例化认证对象：

from airflow.providers.http.operators.http import SimpleHttpOperator
from requests.auth import HTTPBasicAuth

launch_backup_pipeline_task = SimpleHttpOperator(
    task_id="get_task",
    http_conn_id="my_conn",
    method="GET",
    endpoint="my/url",
    auth_type=HTTPBasicAuth,  # 传入类而非实例
    auth_args=('user', 'pass'),  # 传递构造实例的参数
    log_response=True,
)

方法2：自定义HttpHook完全控制认证 #

如果需要绕过Airflow默认的连接配置（避免自定义头被覆盖），可以自定义Hook：

from airflow.providers.http.hooks.http import HttpHook
from requests.auth import HTTPBasicAuth
from airflow.providers.http.operators.http import SimpleHttpOperator

class CustomBasicAuthHook(HttpHook):
    def __init__(self, method='GET', http_conn_id=None, auth=None):
        super().__init__(method, http_conn_id)
        self.auth = auth

    def get_conn(self, headers=None):
        conn = super().get_conn(headers)
        # 强制覆盖连接的认证设置
        conn.auth = self.auth
        return conn

# 使用自定义Hook创建任务
launch_backup_pipeline_task = SimpleHttpOperator(
    task_id="get_task",
    http_conn_id="my_conn",
    method="GET",
    endpoint="my/url",
    hook=CustomBasicAuthHook(auth=HTTPBasicAuth('user', 'pass')),
    log_response=True,
)

方法3：手动构造Basic Auth请求头 #

如果不想依赖requests的认证类，可以手动编码生成Auth头，但要确保http_conn_id对应的连接配置未设置用户名和密码（否则会被Airflow自动生成的头覆盖）：

import base64
from airflow.providers.http.operators.http import SimpleHttpOperator

# 编码用户名密码
credentials = base64.b64encode(b"user:pass").decode('utf-8')
custom_headers = {'Authorization': f'Basic {credentials}'}

launch_backup_pipeline_task = SimpleHttpOperator(
    task_id="get_task",
    http_conn_id="my_conn",
    method="GET",
    endpoint="my/url",
    headers=custom_headers,
    log_response=True,
)

内容的提问来源于stack exchange，提问作者Andrew Coulson