禁用_source时如何用spring-data-opensearch检索存储字段？

索引模板 #

{
  "index_patterns": [
    "request_response-*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "index": {
        "codec": "best_compression"
      }
    },
    "mappings": {
      "_source": {
        "enabled": false
      },
      "properties": {
        "value": {
          "type": "binary",
          "store": true
        }
      }
    },
    "aliases": {
      "request_response": { }
    }
  },
  "priority": 500
}

目标CURL查询 #

需查询匹配指定_id的文档，仅返回单独存储的value字段：

curl -X GET "localhost:9200/request_response-*/_search?stored_fields=value&pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "term": {
      "_id": {
        "value": "092d882489ea6dc4a6d2dc16d97fb72192b17ca97eaaca59961e97597b6ba80c"
      }
    }
  }
}
'

CURL查询返回结果 #

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "request_response-2024-05-09",
        "_type" : "_doc",
        "_id" : "092d882489ea6dc4a6d2dc16d97fb72192b17ca97eaaca59961e97597b6ba80c",
        "_score" : 1.0,
        "fields" : {
          "value" : [
            "U29tZSBiaW5hcnkgYmxvYg=="
          ]
        }
      },
      {
        "_index" : "request_response-2024-05-10",
        "_type" : "_doc",
        "_id" : "092d882489ea6dc4a6d2dc16d97fb72192b17ca97eaaca59961e97597b6ba80c",
        "_score" : 1.0,
        "fields" : {
          "value" : [
            "U29tZSBiaW5hcnkgYmxvYg=="
          ]
        }
      }
    ]
  }
}

尝试的代码及错误 #

尝试的Spring Data OpenSearch代码 #

String documentId = "092d882489ea6dc4a6d2dc16d97fb72192b17ca97eaaca59961e97597b6ba80c";
String[] storedFields = new String[]{"value"};
IndexCoordinates indexCoordinates = IndexCoordinates.of("request_response-*");

NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
    .withQuery(QueryBuilders.termQuery("_id", documentId))
    .withFields(storedFields)
    .build();

SearchHits<KeyValueDoc> searchHits = elasticsearchOperations.search(searchQuery, KeyValueDoc.class, indexCoordinates);
KeyValueDoc document = searchHits.getSearchHit(0).getContent();

抛出的错误 #

Caused by: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=Unable to retrieve the requested [fields] since _source is disabled in the mappings for index [request_response-2024-05-09]]]; nested: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=Unable to retrieve the requested [fields] since _source is disabled in the mappings for index [request_response-2024-05-09]]];
    at org.opensearch.OpenSearchException.innerFromXContent(OpenSearchException.java:550)
    at org.opensearch.OpenSearchException.fromXContent(OpenSearchException.java:461)
    at org.opensearch.OpenSearchException.innerFromXContent(OpenSearchException.java:491)
    at org.opensearch.OpenSearchException.failureFromXContent(OpenSearchException.java:655)
    at org.opensearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:199)
    ... 80 more
Caused by: OpenSearchException[OpenSearch exception [type=illegal_argument_exception, reason=Unable to retrieve the requested [fields] since _source is disabled in the mappings for index [request_response-2024-05-09]]]
    at org.opensearch.OpenSearchException.innerFromXContent(OpenSearchException.java:550)
    at org.opensearch.OpenSearchException.fromXContent(OpenSearchException.java:461)
    at org.opensearch.OpenSearchException.innerFromXContent(OpenSearchException.java:491)
    ... 84 more

正确实现方法 #

问题核心：withFields()方法默认会尝试从_source提取字段，但当前索引已禁用_source，而value是通过store: true单独存储的，必须改用withStoredFields()方法对应OpenSearch的stored_fields参数。

修正后的查询代码 #

String documentId = "092d882489ea6dc4a6d2dc16d97fb72192b17ca97eaaca59961e97597b6ba80c";
String[] storedFields = new String[]{"value"};
IndexCoordinates indexCoordinates = IndexCoordinates.of("request_response-*");

NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
    .withQuery(QueryBuilders.termQuery("_id", documentId))
    .withStoredFields(storedFields) // 替换为withStoredFields
    .build();

SearchHits<KeyValueDoc> searchHits = elasticsearchOperations.search(searchQuery, KeyValueDoc.class, indexCoordinates);
// 遍历处理结果
for (SearchHit<KeyValueDoc> hit : searchHits) {
    KeyValueDoc doc = hit.getContent();
    // 使用doc.getValue()获取base64格式的二进制内容
}

对应的实体类KeyValueDoc #

实体类字段需与索引映射匹配，支持自动处理base64格式的二进制数据：

import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

public class KeyValueDoc {
    @Field(type = FieldType.BINARY, store = true)
    private String value; // 用String接收base64内容，也可使用byte[]自动转换

    // getter和setter
    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

关键说明 #

• withStoredFields()会生成OpenSearch的stored_fields查询参数，专门获取单独存储的字段，不依赖_source。
• 实体类的@Field注解需与索引映射的字段类型、存储属性保持一致。

内容的提问来源于stack exchange，提问作者tuk