如何在LangChain的MongoDBAtlasVectorSearch similarity_search_with_score中正确使用过滤器？

问题背景 #

使用MongoDBAtlasVectorSearch调用similarity_search_with_score查找相似文档时，添加过滤器遇到报错。

现有代码 #

vector_search = MongoDBAtlasVectorSearch(
        collection=client[os.getenv("MONGODB_DB")]["files"],
        embedding=embeddings,
        index_name=os.getenv("ATLAS_VECTOR_SEARCH_INDEX_NAME"),
    )

results = vector_search.similarity_search_with_score(
        query="What are the engagements of the company",
        k=5,
        pre_filter={
            "compound": {
                "filter": [
                    {"equals": {"path": "uploaded_by", "value": chat_owner}},
                    {"in": {"path": "file_name", "values": file_names}},
                ]
            }
        },
    ) 

索引配置 #

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "embedding": {
        "dimensions": 1536,
        "similarity": "cosine",
        "type": "knnVector"
      },
      "file_name": {
        "normalizer": "lowercase",
        "type": "token"
      },
      "uploaded_by": {
        "normalizer": "lowercase",
        "type": "token"
      }
    }
  }
}

第一次报错信息 #

pymongo.errors.OperationFailure: "knnBeta.filter.compound.filter[1].in.value" is required, full error: {'ok': 0.0, 'errmsg': '"knnBeta.filter.compound.filter[1].in.value" is required', 'code': 8, 'codeName': 'UnknownError', '$clusterTime': {'clusterTime': Timestamp(1704804627, 1), 'signature': {'hash': b'\xfa\x15s+Q\x1d\xa86]R\xb2!\x9d\xc5b-G\xce\xa6S', 'keyId': 7283272637088792583}}, 'operationTime': Timestamp(1704804627, 1)}

尝试的另一种写法及报错 #

尝试使用MongoDB原生查询语法：

pre_filter={
    "$and": [
        {"uploaded_by": {"$eq": chat_owner}},
        {"file_name": {"$in": file_names}},
    ]
},

报错：

pymongo.errors.OperationFailure: "knnBeta.filter" one of [autocomplete, compound, embeddedDocument, equals, exists, geoShape, geoWithin, in, knnBeta, moreLikeThis, near, phrase, queryString, range, regex, search, span, term, text, wildcard] must be present, full error: {'ok': 0.0, 'errmsg': '"knnBeta.filter" one of [autocomplete, compound, embeddedDocument, equals, exists, geoShape, geoWithin, in, knnBeta, moreLikeThis, near, phrase, queryString, range, regex, search, span, term, text, wildcard] must be present', 'code': 8, 'codeName': 'UnknownError', '$clusterTime': {'clusterTime': Timestamp(1704802325, 9), 'signature': {'hash': b'`\xd27-\x81+\x16\xd0a\x14\xc7\x99\xa8\x05|Sx?\x0e:', 'keyId': 7283272637088792583}}, 'operationTime': Timestamp(1704802325, 9)}
WARNING:  StatReload detected changes in 'src/routes/chats/chats.py'. Reloading...

解决方案 #

问题根源有两个：

1. Atlas向量搜索的pre_filter必须使用Atlas搜索专属语法，不支持MongoDB原生查询的$and这类操作符。
2. 第一次写法中，in操作符的参数名错误——Atlas搜索语法里，in对应的参数是value（单数），而非values（复数）。

修正后的过滤器代码：

pre_filter={
    "compound": {
        "filter": [
            {"equals": {"path": "uploaded_by", "value": chat_owner}},
            {"in": {"path": "file_name", "value": file_names}},  # 将values改为value
        ]
    }
},

额外注意：

• 确保file_names是一个非空数组，否则可能触发空值相关错误。
• 索引中file_name和uploaded_by配置了小写归一化，过滤时要确保传入的chat_owner和file_names值与存储数据的大小写匹配，或者直接使用归一化后的小写值进行过滤。

内容的提问来源于stack exchange，提问作者colla