Weaviate批量插入记录触发JSONDecodeError问题求助

问题背景 #

使用Weaviate Python客户端批量插入记录时，将对象添加到batch后发送批量请求，始终触发JSONDecodeError。已确认待插入数据可正常JSON序列化，且Schema中定义了Weaviate支持的数组类型。

相关代码与数据 #

批量插入代码 #

client.batch.configure(batch_size=100, dynamic=False, timeout_retries=3,
                       callback=weaviate.util.check_batch_result,
consistency_level=weaviate.data.replication.ConsistencyLevel.ALL)
with client.batch as batch:
     for el_idx, el in enumerate(send_to_weaviate):
         batch.add_data_object(el, "MyClass")

待插入记录示例 #

send_to_weaviate[0]
{'my_id': '3c2466b7e7da201c66f42ea362874343',
 'post_timestamp': ['1644883202000', '1644883242000'],
 'dist_metric': [0, 0]}

Weaviate类Schema #

class_obj = {
        "class": "MyClass",
        "description": "Description",
        "properties": [{
            "dataType": ["text"],
            "description": "ID",
            "name": "my_id"
        },  {
            "dataType": ["text[]"],
            "description": "Timestamps",
            "name": "post_timestamp"
        },  {
            "dataType": ["int[]"],
            "description": "Description",
            "name": "dist_metric"
        }]
    }

报错信息 #

File ~/opt/anaconda3/envs/scripts/lib/python3.9/site-packages/weaviate/batch/crud_batch.py:644, in Batch._create_data(self, data_type, batch_request)
    642     connection_count += 1
    643 else:
--> 644     response_json = response.json()
    645     if (
    646         self._weaviate_error_retry is not None
    647         and batch_error_count < self._weaviate_error_retry.number_retries
    648     ):
    649         batch_to_retry, response_json_successful = self._retry_on_error(
    650             response_json, data_type
    651         )

File ~/opt/anaconda3/envs/scripts/lib/python3.9/site-packages/requests/models.py:975, in Response.json(self, **kwargs)
    971     return complexjson.loads(self.text, **kwargs)
    972 except JSONDecodeError as e:
    973     # Catch JSON-related errors and raise as requests.JSONDecodeError
    974     # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
--> 975     raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

可复现最小示例（weaviate-client v3.22.1，Python3.9） #

import weaviate
client = weaviate.Client(URL_TO_WEAVIATE_ENDPOINT)
class_obj = {
        "class": "MyClass",
        "description": "Description",
        "properties": [{
            "dataType": ["text"],
            "description": "ID",
            "name": "my_id"
        },  {
            "dataType": ["text[]"],
            "description": "Timestamps",
            "name": "post_timestamp"
        },  {
            "dataType": ["int[]"],
            "description": "Description",
            "name": "dist_metric"
        }]
    }
client.schema.create_class(class_obj)
client.batch.configure(batch_size=100, dynamic=False, timeout_retries=3)
# 仅尝试插入1条文档
doc = {'my_id': '3c2466b7e7da201c66f42ea362874343','post_timestamp': ['1644883202000', '1644883242000'], 'dist_metric': [0, 0]}
with client.batch() as batch:
    batch.add_data_object(doc, "MyClass")

问题分析与解决 #

这个JSONDecodeError并非因为数据无法序列化，而是Weaviate服务端返回了空响应或非JSON格式内容，导致客户端解析失败。以下是具体排查与解决步骤：

1. 检查Weaviate服务状态 #

• 访问http://<你的Weaviate地址>/v1/meta，确认返回合法JSON数据，说明服务运行正常。
• 若返回空页面或HTML错误，说明服务未启动或异常，需先修复服务端问题。

2. 捕获并查看服务端响应详情 #

添加代码捕获响应内容，明确服务端返回的实际数据：

try:
    with client.batch() as batch:
        batch.add_data_object(doc, "MyClass")
except Exception as e:
    print("响应状态码:", client.last_response.status_code)
    print("响应文本:", client.last_response.text)

通过响应文本可直接定位问题，比如服务端返回的Schema校验错误、内存不足提示等。

3. 调整超时与批量参数 #

• 延长请求超时时间，避免因超时导致不完整响应：

  client = weaviate.Client(
      URL_TO_WEAVIATE_ENDPOINT,
      timeout_config=(10, 30)  # 连接超时10秒，读取超时30秒
  )
  client.batch.configure(
      batch_size=100,
      dynamic=False,
      timeout_retries=3,
      timeout=30
  )
  

• 临时将batch_size改为1，测试单条插入是否正常，排除批量过大导致的服务端压力问题。

4. 检查Schema兼容性 #

确认Weaviate服务端版本与客户端版本匹配（weaviate-client v3.22.1对应服务端v1.19.x及以上），数组类型写法["text[]"]、["int[]"]在该版本下是合法的；若服务端版本过低，需调整为旧版写法（如["array<string>"]）。

5. 查看服务端日志 #

登录Weaviate服务端，查看日志文件（默认路径为/var/log/weaviate/或容器日志），寻找服务端报错信息，比如Schema校验失败、资源不足等，这些都会导致非JSON响应。

内容的提问来源于stack exchange，提问作者lrthistlethwaite