向Airflow实验性REST API创建DAG Run时传递参数的方法问询

Great question! I totally get why you’d want to pass parameters when triggering a DAG Run via the API—dynamic workflows live on this kind of flexibility, right? And good news: even if the early experimental API docs seemed to say otherwise, you absolutely can pass parameters using the conf field in your POST request payload.

Here's how it works: #

When sending your HTTPS POST to the /api/experimental/dags/<YOUR_DAG_ID>/dag_runs endpoint, include a conf key in your JSON body. This accepts a nested JSON object with any parameters you need to pass to your DAG.

Example with curl:

curl -X POST \
  http://your-airflow-host:8080/api/experimental/dags/my_target_dag/dag_runs \
  -H "Content-Type: application/json" \
  -d '{
        "conf": {
          "customer_id": "12345",
          "process_type": "full",
          "threshold": 95.5
        }
      }'

Accessing the parameters in your DAG:

Once the DAG Run is triggered, you can pull these parameters into your tasks using the dag_run context variable. Here’s a quick PythonOperator example:

from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime

def process_data(**context):
    # Fetch the passed parameters
    run_conf = context["dag_run"].conf
    customer_id = run_conf.get("customer_id")
    process_type = run_conf.get("process_type")
    
    print(f"Processing data for customer {customer_id} with type: {process_type}")
    # Your task logic here...

with DAG(
    dag_id="my_target_dag",
    start_date=datetime(2024, 1, 1),
    schedule_interval=None,
    catchup=False
) as dag:
    process_task = PythonOperator(
        task_id="process_data_task",
        python_callable=process_data,
        provide_context=True
    )

A quick heads-up: #

The experimental API is labeled that way because it’s not as stable as Airflow 2.x’s official stable REST API (v1). The stable API fully supports conf parameters too, with more consistent documentation. But if you’re stuck using the experimental version for now, the conf field works reliably for passing runtime parameters.

内容的提问来源于stack exchange，提问作者Peter Berg