OpenSearch混合查询能否结合布尔过滤条件？

问题背景 #

在OpenSearch中配置集群级搜索管道后，尝试实现文本-向量混合搜索时遇到以下问题：

• 原有逻辑使用Boolean子句做过滤，但Boolean子句无法直接包含hybrid子句
• 需通过metadata.channel_id过滤多客户数据，避免跨客户搜索

索引映射 #

{"hybrid-search-test-index": {"aliases": {},"mappings": {"dynamic": "strict","properties": {"channel_id": {"type": "keyword"},"id": {"type": "keyword"},"lang": {"type": "keyword"},"metadata": {"dynamic": "true","properties": {"article_id": {"type": "keyword"},"faq_id": {"type": "keyword"},"generation_text": {"type": "keyword"},"image_chunk_ids": {"type": "keyword"},"image_hash": {"type": "keyword"},"image_url": {"type": "keyword"},"revision_id": {"type": "keyword"},"screenshot": {"type": "boolean"},"space_id": {"type": "keyword"},"text_chunk_ids": {"type": "keyword"},"text_hash": {"type": "keyword"}}},"texts": {"properties": {"en": {"type": "text","analyzer": "en_analyzer"},"etc": {"type": "text","analyzer": "en_analyzer"},"ja": {"type": "text","analyzer": "ja_analyzer"},"ko": {"type": "text","analyzer": "ko_analyzer"},"subtype": {"type": "keyword"}}},"type": {"type": "keyword"},"vectors": {"type": "nested","properties": {"model_name": {"type": "keyword"},"model_version": {"type": "keyword"},"subtype": {"type": "keyword"},"values": {"type": "knn_vector","dimension": 1024,"method": {"engine": "lucene","space_type": "cosinesimil","name": "hnsw","parameters": {}}}}}}},"settings": {"index": {"replication": {"type": "DOCUMENT"},"blocks": {"write": "true"},"provided_name": "hybrid-search-test-index","knn": "true","creation_date": "1742202362678","analysis": {"analyzer": {"ja_analyzer": {"filter": ["kuromoji_baseform","kuromoji_part_of_speech","kuromoji_readingform","kuromoji_stemmer","lowercase"],"type": "custom","tokenizer": "kuromoji_tokenizer"},"ko_analyzer": {"filter": ["nori_readingform","lowercase","nori_part_of_speech"],"type": "custom","tokenizer": "nori_tokenizer"},"en_analyzer": {"type": "standard"}}},"number_of_replicas": "1","uuid": "ypIc3md0T8esjj6clYUnQw","version": {"created": "136397827","upgraded": "136397827"},"routing": {"allocation": {"initial_recovery": {"_id": null}}},"number_of_shards": "1","routing_partition_size": "1","resize": {"source": {"name": "table-rag-test-bullsone","uuid": "D_3bxlrIQfOsTACAXmrzzg"}}}}}}

用户原始查询 #

POST /hybrid-search-test-index/_search?search_pipeline=nlp-search-pipeline
{"size": 10,"_source": {"exclude": ["vectors.values"]},"query": {"bool": {"filter": {"nested": {"path": "vectors","query": {"bool": {"filter": [{"term": {"vectors.subtype": "cell"}},{"term": {"vectors.model_name": "embedding"}},{"term": {"vectors.model_version": "240718"}}]}}}},"hybrid": {"queries": [{"match": {"texts.ko": {"query": "체어","boost": 1.0}}},{"nested": {"path": "vectors","query": {"knn": {"vectors.values": {"vector": [1],"k": 10,"boost": 1.0}}}}}]}}}

可行解决方案 #

方法1：将过滤条件嵌入每个hybrid子查询 #

把全局过滤条件（包括metadata.channel_id和vectors相关过滤）添加到hybrid的每个子查询的bool过滤中，确保所有子查询都先执行过滤再搜索。

修改后的查询示例：

POST /hybrid-search-test-index/_search?search_pipeline=nlp-search-pipeline
{
  "size": 10,
  "_source": {"exclude": ["vectors.values"]},
  "query": {
    "hybrid": {
      "queries": [
        {
          "bool": {
            "must": [{"match": {"texts.ko": {"query": "체어","boost": 1.0}}}],
            "filter": [
              {"term": {"metadata.channel_id": "指定客户ID"}},
              {"nested": {
                "path": "vectors",
                "query": {"bool": {"filter": [
                  {"term": {"vectors.subtype": "cell"}},
                  {"term": {"vectors.model_name": "embedding"}},
                  {"term": {"vectors.model_version": "240718"}}
                ]}}
              }}
            ]
          }
        },
        {
          "bool": {
            "must": [{"nested": {
              "path": "vectors",
              "query": {"knn": {"vectors.values": {"vector": [1],"k": 10,"boost": 1.0}}}
            }}],
            "filter": [
              {"term": {"metadata.channel_id": "指定客户ID"}},
              {"nested": {
                "path": "vectors",
                "query": {"bool": {"filter": [
                  {"term": {"vectors.subtype": "cell"}},
                  {"term": {"vectors.model_name": "embedding"}},
                  {"term": {"vectors.model_version": "240718"}}
                ]}}
              }}
            ]
          }
        }
      ]
    }
  }
}

方法2：用function_score包装hybrid，外层bool添加全局过滤 #

在外层bool的filter中定义统一的过滤条件，must中用function_score包装hybrid查询，这样过滤会先作用于整个索引，再执行混合搜索。

修改后的查询示例：

POST /hybrid-search-test-index/_search?search_pipeline=nlp-search-pipeline
{
  "size": 10,
  "_source": {"exclude": ["vectors.values"]},
  "query": {
    "bool": {
      "filter": [
        {"term": {"metadata.channel_id": "指定客户ID"}},
        {"nested": {
          "path": "vectors",
          "query": {"bool": {"filter": [
            {"term": {"vectors.subtype": "cell"}},
            {"term": {"vectors.model_name": "embedding"}},
            {"term": {"vectors.model_version": "240718"}}
          ]}}
        }}
      ],
      "must": [
        {
          "function_score": {
            "query": {
              "hybrid": {
                "queries": [
                  {"match": {"texts.ko": {"query": "체어","boost": 1.0}}},
                  {"nested": {
                    "path": "vectors",
                    "query": {"knn": {"vectors.values": {"vector": [1],"k": 10,"boost": 1.0}}}
                  }}
                ]
              }
            }
          }
        }
      ]
    }
  }
}

方案选择 #

• 方法1适合需要为不同hybrid子查询设置不同过滤规则的场景
• 方法2更简洁，适用于所有子查询共享同一全局过滤条件的场景，性能更优

内容的提问来源于stack exchange，提问作者Sean