Docker部署Airflow出现alembic无法定位修订版本a13f7613ad25报错如何解决

问题现象 #

采用Docker部署Airflow，执行初始化数据库操作时抛出如下异常：

webserver_1  | The above exception was the direct cause of the following exception:
webserver_1  | 
webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 8, in <module>
webserver_1  |     sys.exit(main())
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
webserver_1  |     args.func(args)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
webserver_1  |     return func(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
webserver_1  |     db.initdb()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 549, in initdb
webserver_1  |     upgradedb()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 684, in upgradedb
webserver_1  |     command.upgrade(config, 'heads')
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 294, in upgrade
webserver_1  |     script.run_env()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 490, in run_env
webserver_1  |     util.load_python_file(self.dir, "env.py")
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
webserver_1  |     module = load_module_py(module_id, path)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
webserver_1  |     spec.loader.exec_module(module)
webserver_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
webserver_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 108, in <module>
webserver_1  |     run_migrations_online()
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
webserver_1  |     context.run_migrations()
webserver_1  |   File "<string>", line 8, in run_migrations
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
webserver_1  |     self.get_context().run_migrations(**kw)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 549, in run_migrations
webserver_1  |     for step in self._migrations_fn(heads, self):
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 283, in upgrade
webserver_1  |     return script._upgrade_revs(revision, rev)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 370, in _upgrade_revs
webserver_1  |     for script in reversed(list(revs))
webserver_1  |   File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
webserver_1  |     self.gen.throw(type, value, traceback)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 203, in _catch_revision_errors
webserver_1  |     compat.raise_(util.CommandError(resolution), from_=re)
webserver_1  |   File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 296, in raise_
webserver_1  |     raise exception
webserver_1  | alembic.util.exc.CommandError: Can't locate revision identified by 'a13f7613ad25'

该问题仅在本地环境复现，其他同事的运行环境均无此问题，排查难度较高。

根因说明 #

这个错误是Airflow依赖的数据库迁移工具Alembic抛出的，本质是当前你本地Airflow使用的数据库中alembic_version表存储的迁移版本号a13f7613ad25，在你当前运行的Airflow版本的迁移脚本文件中不存在，导致迁移流程无法继续。

定位步骤 #

• 排查本地是否有残留的Airflow数据库存储：你本地大概率之前运行过更高版本、或者定制分支版本的Airflow，对应生成的数据库已经执行过新版本的迁移脚本，留下了对应版本号记录。而你现在运行的Airflow版本较低，没有对应这个版本号的迁移脚本。其他同事都是首次部署，用的全新数据库，因此不会触发该问题。
• 检查本地Docker存储卷残留：执行docker volume ls查看是否有未清理的Airflow PostgreSQL/MySQL存储卷，旧卷被新启动的容器直接挂载，导致复用了旧数据库。
• 对比本地Airflow镜像和同事使用的镜像是否一致：执行docker images | grep airflow查看本地镜像ID，和同事的镜像ID做对比，避免你本地拉取的镜像版本和其他人不一致，缺失对应迁移文件。

修复方案 #

• 若本地数据库无需要保留的历史数据，直接清理残留存储卷即可：先停止所有Airflow相关容器，执行docker volume rm <对应的Airflow数据库卷名>删除旧卷，再重新启动服务执行初始化操作即可正常运行。
• 若需要保留数据库数据，可以手动修正版本号：首先查询你当前使用的Airflow版本对应的最新迁移版本号，登录Airflow使用的数据库执行UPDATE alembic_version SET version_num = '<对应版本的正确迁移版本号>';，之后再执行数据库升级操作即可。
• 若确认是镜像版本不一致导致的问题，删除本地的Airflow镜像，重新拉取和同事一致的镜像版本后重新部署即可。

内容的提问来源于stack exchange，提问作者Canovice