TriggerDagRunOperator触发的子DAG仅排队不运行问题排查

父DAG（master）通过TriggerDagRunOperator触发子DAG（dag_tabular）后，子DAG始终处于排队状态无法执行。但移除TriggerDagRunOperator的wait_for_completion=True参数后，父子DAG均可正常运行。

已做排查 #

• 排查资源锁或Worker资源问题：检查Airflow配置中的workers、worker_concurrency、max_active_runs_per_dag参数，未发现异常。
• 该DAG流程在Google Cloud Compose环境下可正常执行。

父DAG代码（parent_dag.py） #

from airflow import DAG
from datetime import datetime, timedelta

from airflow.decorators import dag
from airflow.operators.trigger_dagrun import TriggerDagRunOperator
from airflow.operators.python import PythonOperator


def setup_environment(ti, **context):
    print("parent dag")


with DAG(
    dag_id="master",
    default_args={"owner": "airflow"},
    start_date=datetime(2022, 8, 8),
    schedule_interval=None,
    tags=["master"],
    catchup=False,
) as dag:

    setup = PythonOperator(
        task_id="setup",
        python_callable=setup_environment,
        provide_context=True,
    )

    dag_tabular = TriggerDagRunOperator(
        task_id="trigger_tabular",
        trigger_dag_id="dag_tabular",
        trigger_run_id="{{run_id}}",
        wait_for_completion=True,
    )
setup >> dag_tabular

子DAG代码（tabular.py） #

from airflow import DAG
from datetime import datetime, timedelta
from airflow.operators.python import PythonOperator
import time


def setup_environment(ti, **context):
    print("tabular dag")


def stopper(ti, **context):
    time.sleep(60)
    print("done with tabular")


with DAG(
    dag_id="dag_tabular",
    default_args={"owner": "airflow"},
    start_date=datetime(2022, 8, 8),
    schedule_interval=None,
    tags=["tabular"],
    catchup=False,
) as dag:
    setup = PythonOperator(
        task_id="setup", python_callable=setup_environment, provide_context=True
    )
    stopper = PythonOperator(
        task_id="stop_tabular", python_callable=stopper, provide_context=True
    )

setup >> stopper

内容的提问来源于stack exchange，提问作者shubh gupta