Airflow任务启动24小时后被终止：令牌过期问题求助

任务运行时长达到24小时时会被立即终止，日志提示令牌过期。

• Airflow 3.1.6（运行于Docker环境）
• Celery executor（worker_concurrency=16，worker_autoscale="2,32"）
• auth_manager = airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
• fab -> auth_backends = airflow.providers.fab.auth_manager.api.auth.backend.basic_auth,airflow.providers.fab.auth_manager.api.auth.backend.session
• api_auth -> jwt_expiration_time = 2628000（1个月）
• api_auth -> jwt_cli_expiration_time = 3600（1小时）
• api_auth -> jwt_leeway = 10
• execution_api -> jwt_expiration_time = 600（10分钟）

[2026-02-19T12:10:34.423556Z] {celery_executor_utils.py:154} INFO - [a6915c74-3b86-46df-a0f2-2bd3848a98f4] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('019c70a7-fb7c-7480-84d0-7232151770ed'), dag_version_id=UUID('019c4721-09cb-7d79-8402-e3925d3ab5b2'), task_id='mt68-gbr-crawler', dag_id='selbstaendige-gbr-crawler', run_id='manual__2026-02-18T12:09:40+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}) dag_rel_path=PurePosixPath('selbstaendige/gbr_crawler.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=selbstaendige-gbr-crawler/run_id=manual__2026-02-18T12:09:40+00:00/task_id=mt68-gbr-crawler/attempt=1.log' type='ExecuteTask'
[2026-02-19T12:10:34.425926Z] {supervisor.py:1975} INFO - Secrets backends loaded for worker count=1 backend_classes=['EnvironmentVariablesBackend']
[2026-02-19T12:10:34.448869Z] {supervisor.py:710} INFO - Process exited pid=123088 exit_code=-9 signal_sent=SIGKILL
[2026-02-19T12:10:34.462386Z] {trace.py:285} ERROR - Task execute_workload[a6915c74-3b86-46df-a0f2-2bd3848a98f4] raised unexpected: ServerResponseError('Invalid auth token: Signature has expired')
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/celery/app/trace.py", line 479, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/celery/app/trace.py", line 779, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/celery/executors/celery_executor_utils.py", line 162, in execute_workload
    supervise(
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 1984, in supervise
    process = ActivitySubprocess.start(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 955, in start
    proc._on_child_started(ti=what, dag_rel_path=dag_rel_path, bundle_info=bundle_info)
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 966, in _on_child_started
    ti_context = self.client.task_instances.start(ti.id, self.pid, start_date)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py", line 215, in start
    resp = self.client.patch(f"task-instances/{id}/run", content=body.model_dump_json())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 1218, in patch
    return self.request(
           ^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 338, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 477, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 378, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 400, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 480, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py", line 887, in request
    return super().request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 999, in _send_handling_redirects
    raise exc
  File "/home/airflow/.local/lib/python3.12/site-packages/httpx/_client.py", line 982, in _send_handling_redirects
    hook(response)
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py", line 186, in raise_on_4xx_5xx_with_note
    return get_json_error(response) or response.raise_for_status()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/api/client.py", line 176, in get_json_error
    raise err
airflow.sdk.api.client.ServerResponseError: Invalid auth token: Signature has expired
Correlation-id=019c75cf-2041-7fc7-a554-970c6a610cb3
INFO:     172.25.0.5:52116 - "GET /log/dag_id%3Dselbstaendige-gbr-crawler/run_id%3Dmanual__2026-02-18T12%3A09%3A40%2B00%3A00/task_id%3Dmt68-gbr-crawler/attempt%3D1.log HTTP/1.1" 200 OK
[2026-02-19T12:10:36.186345Z] {supervisor.py:1115} ERROR - Server indicated the task shouldn't be running anymore detail={'detail': {'reason': 'not_running', 'message': 'TI is no longer in the running state and task should terminate', 'current_state': 'failed'}} status_code=409 ti_id=UUID('019c70a7-fb7c-7480-84d0-7232151770ed')
[2026-02-19T12:10:41.192109Z] {supervisor.py:718} WARNING - Process did not terminate in time; escalating pid=3873 signal=SIGTERM
[2026-02-19T12:10:41.201091Z] {supervisor.py:710} INFO - Process exited pid=3873 exit_code=-9 signal_sent=SIGKILL

从日志和配置来看，核心问题是execution_api的JWT令牌过期时间过短。当前execution_api -> jwt_expiration_time仅设置为10分钟，远低于任务24小时的运行时长。当worker在任务运行过程中调用Airflow Execution API更新任务状态时，初始生成的令牌已过期，导致API调用失败，触发任务终止。

解决步骤： #

1. 修改Airflow配置文件，调整execution_api下的jwt_expiration_time参数，设置为大于任务最长运行时长的值。例如，若任务最长运行24小时，可设置为：

   [execution_api]
   jwt_expiration_time = 86400
   

2. 重启Airflow所有相关服务（webserver、scheduler、celery workers），确保配置生效。
3. 运行长时长任务进行验证，确认令牌过期问题已解决。

额外建议： #

• 若存在更长运行时长的任务，可根据实际需求进一步延长令牌有效期，比如设置为604800（7天），避免频繁调整。
• 检查其他API相关的令牌过期配置，确保所有涉及长任务的API令牌有效期覆盖任务全运行周期。

内容的提问来源于stack exchange，提问作者Alkis Mavridis