Airflow GUI创建连接报错：找不到airflow.providers.cncf模块

环境信息 #

• Apache Airflow版本：2.5.0
• 操作系统：AWS EC2 GNU Linux 3.10.0-1160.81.1.el7.x86_64
• Python版本：3.9.5

问题现象 #

进入Airflow Web界面【Admin -> Connections】，点击+号创建新连接或编辑现有连接时，触发ModuleNotFoundError: No module named 'airflow.providers.cncf'错误，无法正常操作连接。

关键错误日志 #

ModuleNotFoundError: No module named 'airflow.providers.cncf'

完整Traceback：

Traceback (most recent call last): File
"/home/airflow/.local/lib/python3.9/site-packages/flask/app.py", line
    > 2528, in wsgi_app response = self.full_dispatch_request() File
"/home/airflow/.local/lib/python3.9/site-packages/flask/app.py", line
1825, in full_dispatch_request rv = self.handle_user_exception(e) File
"/home/airflow/.local/lib/python3.9/site-packages/flask/app.py", line
1823, in full_dispatch_request rv = self.dispatch_request() File
"/home/airflow/.local/lib/python3.9/site-packages/flask/app.py", line
1799, in dispatch_request return
self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File
"/usr/local/lib/python3.9/site-packages/airflow/www/decorators.py",
line 125, in wrapper return f(*args, **kwargs) File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py",
line 133, in wraps return f(self, *args, **kwargs) File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/views.py",
line 594, in add return self.render_template( File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/baseviews.py",
line 322, in render_template return render_template( File
"/home/airflow/.local/lib/python3.9/site-packages/flask/templating.py",
line 147, in render_template return _render(app, template, context)
File
"/home/airflow/.local/lib/python3.9/site-packages/flask/templating.py",
line 130, in _render rv = template.render(context) File
"/home/airflow/.local/lib/python3.9/site-packages/jinja2/environment.py",
line 1301, in render self.environment.handle_exception() File
"/home/airflow/.local/lib/python3.9/site-packages/jinja2/environment.py",
line 936, in handle_exception raise
rewrite_traceback_stack(source=source) File
"/usr/local/lib/python3.9/site-packages/airflow/www/templates/airflow/conn_create.html",
line 20, in top-level template code {% extends
'appbuilder/general/model/add.html' %} File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/templates/appbuilder/general/model/add.html",
line 2, in top-level template code {% import
'appbuilder/general/lib.html' as lib %} File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/templates/appbuilder/base.html",
line 1, in top-level template code {% extends base_template %} File
"/usr/local/lib/python3.9/site-packages/airflow/www/templates/airflow/main.html",
line 21, in top-level template code {% from 'airflow/_messages.html'
import show_message %} File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html",
line 2, in top-level template code {% import 'appbuilder/baselib.html'
as baselib %} File
"/usr/local/lib/python3.9/site-packages/flask_appbuilder/templates/appbuilder/init.html",
line 50, in top-level template code {% block tail %} File
"/usr/local/lib/python3.9/site-packages/airflow/www/templates/airflow/conn_create.html",
line 31, in block 'tail' <script id="field_behaviours"
type="text/json">{{ widgets["add"].field_behaviours }}</script> File
"/home/airflow/.local/lib/python3.9/site-packages/jinja2/environment.py",
line 485, in getattr return getattr(obj, attribute) File
"/usr/local/lib/python3.9/functools.py", line 969, in get val =
self.func(instance) File
"/usr/local/lib/python3.9/site-packages/airflow/www/views.py", line
4217, in field_behaviours return
json.dumps(ProvidersManager().field_behaviours) File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 957, in field_behaviours self.initialize_providers_hooks() File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 327, in wrapped_function func(*args, **kwargs) File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 413, in initialize_providers_hooks
self.initialize_providers_list() File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 327, in wrapped_function func(*args, **kwargs) File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 390, in initialize_providers_list
self._discover_all_providers_from_packages() File
"/usr/local/lib/python3.9/site-packages/airflow/providers_manager.py",
line 462, in _discover_all_providers_from_packages provider_info =
entry_point.load()() File
"/usr/local/lib/python3.9/site-packages/importlib_metadata/__init__.py",
line 207, in load module = import_module(match.group('module')) File
"/usr/local/lib/python3.9/importlib/__init__.py", line 127, in
import_module return _bootstrap._gcd_import(name[level:], package,
level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in
_find_and_load File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1030, in
_gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 228, in
_call_with_frames_removed File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 984, in
_find_and_load_unlocked ModuleNotFoundError: No module named 'airflow.providers.cncf'

解决方案 #

方案1：安装CNCF Provider包 #

airflow.providers.cncf是Airflow官方的CNCF集成提供者，对应包名为apache-airflow-providers-cncf-kubernetes，安装适配Airflow 2.5.0的版本：

pip install apache-airflow-providers-cncf-kubernetes==5.1.0

（注：Airflow 2.5.0与该版本兼容）

安装完成后验证模块是否加载成功：

python -c "import airflow.providers.cncf.kubernetes; print('模块加载成功')"

最后重启Airflow服务：

airflow webserver restart
airflow scheduler restart

方案2：禁用不需要的CNCF Provider #

若环境中不需要Kubernetes等CNCF相关功能，可修改airflow.cfg配置文件，在[core]章节添加：

disable_providers = airflow.providers.cncf

修改后重启Airflow服务即可生效。

内容的提问来源于stack exchange，提问作者Austin Jackson