Docker部署Apache Superset无法连接Dremio求助

• 部署方式：通过docker-compose-non-dev.yml部署Apache Superset，已在superset/docker/requirements-local.txt中添加sqlalchemy-dremio以支持Dremio
• Dremio部署：用命令docker run -p 9047:9047 -p 31010:31010 -p 45678:45678 -p 32010:32010 dremio/dremio-oss单独启动Dremio OSS容器，已在localhost:9047网页界面创建账号
• 连接问题：在Superset中添加Dremio数据库并测试连接时失败，使用的连接字符串为dremio+flight://dremio:dremio123@host.docker.internal:32010/dremio;SSL=0
• 排除项：使用GitHub上的Python脚本python example.py -host host.docker.internal -query 'SELECT 1'，无论从宿主机（用localhost）还是Superset容器内（用host.docker.internal）都能成功连接Dremio，说明网络配置正常且sqlalchemy-dremio已正确安装

Superset容器错误日志 #

2022-09-30 16:34:09,635:WARNING:superset.views.base:SupersetErrorsException
Traceback (most recent call last):
  File "/app/superset/databases/commands/test_connection.py", line 123, in run
    raise DBAPIError(None, None, None)
sqlalchemy.exc.DBAPIError: (builtins.NoneType) None
(Background on this error at: https://sqlalche.me/e/14/dbapi)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 114, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 111, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1572, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 244, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 84, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/databases/api.py", line 708, in test_connection
    TestConnectionDatabaseCommand(item).run()
  File "/app/superset/databases/commands/test_connection.py", line 148, in run
    raise DatabaseTestConnectionFailedError(errors) from ex
superset.databases.commands.exceptions.DatabaseTestConnectionFailedError: [SupersetError(message='(builtins.NoneType) None\n(Background on this error at: https://sqlalche.me/e/14/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Dremio', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
***************
['UID=dremio', 'PWD=dremio123', 'HOST=host.docker.internal', 'PORT=32010', 'Schema=dremio', 'SSL=0']
***************

排查解决方法 #

1. 调整连接字符串格式
   将分号分隔的参数改为URL查询参数形式，部分SQLAlchemy驱动对参数分隔符解析存在差异：

   dremio+flight://dremio:dremio123@host.docker.internal:32010/dremio?SSL=0
   

2. 验证并升级驱动版本
   进入Superset容器，执行pip show sqlalchemy-dremio确认驱动版本，若版本较低，升级到最新兼容版本：

   pip install --upgrade sqlalchemy-dremio
   

   升级后重启Superset容器。

3. 手动测试连接与元数据查询
   在Superset容器内执行以下Python代码，直接测试连接并尝试获取元数据，暴露具体错误：

   from sqlalchemy import create_engine
   from sqlalchemy.inspect import inspect
   
   engine = create_engine("dremio+flight://dremio:dremio123@host.docker.internal:32010/dremio?SSL=0")
   connection = engine.connect()
   # 尝试获取表列表
   inspector = inspect(engine)
   print(inspector.get_table_names())
   connection.close()
   

4. 检查Dremio账号权限
   确认Dremio账号dremio拥有访问dremio schema的权限，包括查看元数据、执行查询的权限，可在Dremio界面中为该账号添加对应权限。

内容的提问来源于stack exchange，提问作者Max test