Docker容器内Airflow指定版本与实际不符致容器重启,求原因
问题描述
我尝试使用官方docker-compose文件运行Airflow,但airflow-webserver、airflow-scheduler、airflow-worker、airflow-triggerer容器持续重启。查看容器日志,显示如下警告与错误:
airflow_drck-airflow-webserver-1 | /home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py:408 FutureWarning: The auth_backends setting in [api] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
airflow_drck-airflow-webserver-1 | /home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_config.py:1001 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
airflow_drck-airflow-webserver-1 | ERROR: You need to upgrade the database. Please runairflow db upgrade. Make sure the command is run using Airflow version 2.6.1.
问题核心矛盾:docker-compose文件中指定的Airflow版本为2.3.3,但进入容器执行airflow version命令时,显示版本为2.6.1:
default@5c1f84edc227:/opt/airflow$ airflow version
/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py:408 FutureWarning: The auth_backends setting in [api] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_config.py:1001 DeprecationWarning: The namespace option in [kubernetes] has been moved to the namespace option in [kubernetes_executor] - the old setting has been used, but please update your config.
2.6.1
版本不符的原因
- 镜像标签不明确:如果docker-compose文件中使用的是
apache/airflow:latest或模糊版本标签(如2),Docker会自动拉取对应分支的最新稳定镜像,即2.6.1,而非指定的2.3.3。 - 本地镜像缓存冲突:此前可能已拉取过Airflow 2.6.1镜像,docker-compose启动时优先使用本地现有镜像,忽略了compose文件中的版本设置(若标签存在歧义)。
- compose文件版本不匹配:下载的官方docker-compose文件为新版,默认配置了更高版本的Airflow镜像,与你预期的2.3.3版本不符。
解决办法
- 明确指定完整版本镜像:修改docker-compose.yml中所有Airflow服务的镜像标签为
apache/airflow:2.3.3,确保webserver、scheduler、worker等所有组件统一使用该版本。 - 清理旧镜像并重新部署:执行以下命令清理本地冲突镜像,再重新启动服务:
docker rmi apache/airflow:2.6.1 apache/airflow:latest docker-compose down -v docker-compose up -d - 升级数据库适配新版本:若选择保留2.6.1版本,需先执行数据库升级命令,再重启服务:
docker-compose run --rm webserver airflow db upgrade docker-compose restart
内容的提问来源于stack exchange,提问作者irbiss

