Weaviate自定义向量失效：上传1024维向量却存为384维

操作步骤 #

• 本地部署Weaviate的docker-compose.yml配置：

version: '3.4'
services:
  weaviate:
    image: cr.weaviate.io/semitechnologies/weaviate:1.25.0
    restart: on-failure:0
    ports:
    - 8080:8080
    - 50051:50051
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-transformers
      ENABLE_MODULES: text2vec-transformers
      TRANSFORMERS_INFERENCE_API: http://t2v-transformers:8080
      CLUSTER_HOSTNAME: 'node1'
  t2v-transformers:
    image: semitechnologies/transformers-inference:sentence-transformers-multi-qa-MiniLM-L6-cos-v1
    environment:
      ENABLE_CUDA: 0

• 创建legal_sections Collection时，设置content属性跳过向量化：

client.collections.create(name = "legal_sections", 
                          properties = [wvc.config.Property(name = "content",
                                                           description = "The actual section chunk that the answer is to be extracted from",
                                                           data_type = wvc.config.DataType.TEXT,
                                                           index_searchable = True,
                                                           index_filterable = True,
                                                           skip_vectorization = True,
                                                           vectorize_property_name = False)])

• 生成并上传自行编码的1024维向量：

upserts = []
for content, vector in zip(docs, embeddings.encode(docs)):
    upserts.append(wvc.data.DataObject(
        properties = {
            'content':content
        },
        vector = vector
    ))

client.collections.get("Legal_sections").data.insert_many(upserts)

• 确认上传前向量维度为1024：

upserts[0].vector.shape
output:
(1024,)

• 查询存储对象时，发现向量维度变为384：

coll = client.collections.get("legal_sections")

for i in coll.iterator():
    print(i.uuid)
    break
output:
386be699-71de-4bad-9022-31173b9df8d2

coll.query.fetch_object_by_id('386be699-71de-4bad-9022-31173b9df8d2', include_vector=True).vector['default'].__len__()
output:
384

问题原因 #

你只设置了属性级别的跳过向量化，但没有关闭Collection级别的自动向量化。部署配置里指定了DEFAULT_VECTORIZER_MODULE: text2vec-transformers，配套的t2v-transformers服务使用的模型sentence-transformers-multi-qa-MiniLM-L6-cos-v1输出维度正好是384。

虽然content属性不会被自动向量化，但Weaviate默认会对整个对象启用指定的向量器生成向量，直接覆盖了你上传的自定义1024维向量。

解决方法 #

创建Collection时，必须明确关闭自动向量化功能，添加vectorizer_config配置为none()：

client.collections.create(
    name = "legal_sections", 
    properties = [wvc.config.Property(
        name = "content",
        description = "The actual section chunk that the answer is to be extracted from",
        data_type = wvc.config.DataType.TEXT,
        index_searchable = True,
        index_filterable = True,
        skip_vectorization = True,
        vectorize_property_name = False
    )],
    vectorizer_config=wvc.config.Config.Vectorizer.none()  # 关键：禁用自动向量化
)

或者简化写法：

client.collections.create(
    name = "legal_sections", 
    properties = [...],  # 原属性配置
    vectorizer="none"
)

这样Weaviate就不会自动生成向量，会完整保留你上传的1024维自定义向量。

内容的提问来源于stack exchange，提问作者figs_and_nuts