如何在Python中连接云部署Druid并执行查询？

问题描述 #

本地Druid实例可通过Python的druid_query库正常执行查询，但连接云生产环境Druid时触发json.decoder.JSONDecodeError错误。当前在AWS机器上运行脚本，Druid部署在另一台机器，日常需通过VPN访问，用cURL调用broker API仅本地环境可行。

原代码：

from druid_query.client import Client
from druid_query.queries import Sql

client = Client(sql_endpoint='http://druid_url.vpc/console.html#query')

query = Sql('SELECT * FROM table1 LIMIT 5')

result = client.execute(query)

print(result)

报错信息：

Traceback (most recent call last):
  File "/Users/pankaj/pani/scripts/druid.py", line 8, in <module>
    result = client.execute(query)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/druid_query/client.py", line 50, in execute
    return process_response(resp, query)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/druid_query/client.py", line 94, in process_response
    return resp.json()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_models.py", line 742, in json
    return jsonlib.loads(self.content.decode(encoding), **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

核心问题与修复方案 #

1. 修正SQL端点地址 #

你错误地将Druid控制台的网页地址（console.html#query）传给了sql_endpoint参数，该参数需要的是Druid Broker的SQL API接口地址，正确格式为http://druid_broker_host:port/druid/v2/sql/。

修改后的代码：

from druid_query.client import Client
from druid_query.queries import Sql

# 替换为实际的Druid Broker地址和端口（默认端口为8888）
client = Client(sql_endpoint='http://druid_url.vpc:8888/druid/v2/sql/')

query = Sql('SELECT * FROM table1 LIMIT 5')

result = client.execute(query)

print(result)

2. 验证网络连通性 #

• 确认AWS机器已成功连接VPN，可通过ping druid_url.vpc或telnet druid_url.vpc 8888测试端口连通性。
• 若VPN正常但仍无法访问，检查Druid Broker的安全组/防火墙规则，确认是否允许AWS机器的IP访问8888端口（Druid默认Broker端口）。

3. 检查API响应格式 #

修正端点后若仍报错，先手动用cURL测试API是否返回合法JSON：

curl -X POST -H "Content-Type: application/json" -d '{"query":"SELECT * FROM table1 LIMIT 5"}' http://druid_url.vpc:8888/druid/v2/sql/

若返回HTML或非JSON内容，说明端点仍错误，或Druid Broker配置存在问题。

4. 认证与权限检查 #

若云环境Druid开启了认证（如Basic Auth），需在Client初始化时传入认证信息：

client = Client(
    sql_endpoint='http://druid_url.vpc:8888/druid/v2/sql/',
    auth=('username', 'password')
)

内容的提问来源于stack exchange，提问作者Pankaj Kumar