如何通过Java High Level API从Elasticsearch多异构嵌套索引获取数据

Hey there! I totally get where you're coming from—Elasticsearch's lack of native joins can feel limiting when you're used to relational databases, but we've got solid workarounds using the Java High Level REST API. Let's break down the most practical approaches for your scenario:

Since Elasticsearch isn't built for relational-style joins, the standard solution is to fetch data from each index separately and stitch it together in your Java code. This gives you full control over how you associate the nested structures.

Here's how to extend your example to implement this:

import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;

import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class MultiIndexAssociationExample {

    public List<Map<String, Object>> getAssociatedData(RestHighLevelClient client) throws IOException {
        // Step 1: Fetch data from index_1 to get identifiers for association
        SearchRequest index1Request = new SearchRequest("index_1");
        SearchSourceBuilder index1Source = new SearchSourceBuilder();
        index1Source.query(QueryBuilders.termQuery("user", "kimchy"));
        // Only fetch fields needed to link to other indices (e.g., a shared order ID)
        index1Source.fetchSource(new String[]{"related_order_id"}, null);
        SearchResponse index1Response = client.search(index1Request, RequestOptions.DEFAULT);

        // Extract all relevant IDs for subsequent queries
        List<String> relatedOrderIds = new ArrayList<>();
        for (SearchHit hit : index1Response.getHits().getHits()) {
            String orderId = hit.getSourceAsMap().get("related_order_id").toString();
            relatedOrderIds.add(orderId);
        }

        List<Map<String, Object>> finalCombinedResults = new ArrayList<>();
        if (!relatedOrderIds.isEmpty()) {
            // Step 2: Fetch matching data from index_2 and index_3
            // Query index_2 for order details
            SearchRequest index2Request = new SearchRequest("index_2");
            SearchSourceBuilder index2Source = new SearchSourceBuilder();
            index2Source.query(QueryBuilders.termsQuery("order_id", relatedOrderIds));
            SearchResponse index2Response = client.search(index2Request, RequestOptions.DEFAULT);

            // Query index_3 for related product data
            SearchRequest index3Request = new SearchRequest("index_3");
            SearchSourceBuilder index3Source = new SearchSourceBuilder();
            index3Source.query(QueryBuilders.termsQuery("linked_order_id", relatedOrderIds));
            SearchResponse index3Response = client.search(index3Request, RequestOptions.DEFAULT);

            // Step 3: Create lookup maps for fast association
            Map<String, Map<String, Object>> index2Lookup = new HashMap<>();
            for (SearchHit hit : index2Response.getHits().getHits()) {
                String orderId = hit.getSourceAsMap().get("order_id").toString();
                index2Lookup.put(orderId, hit.getSourceAsMap());
            }

            Map<String, List<Map<String, Object>>> index3Lookup = new HashMap<>();
            for (SearchHit hit : index3Response.getHits().getHits()) {
                String orderId = hit.getSourceAsMap().get("linked_order_id").toString();
                index3Lookup.computeIfAbsent(orderId, k -> new ArrayList<>()).add(hit.getSourceAsMap());
            }

            // Step 4: Stitch together data from all indices
            for (SearchHit index1Hit : index1Response.getHits().getHits()) {
                Map<String, Object> combinedData = new HashMap<>(index1Hit.getSourceAsMap());
                String orderId = combinedData.get("related_order_id").toString();
                // Add matching data from index_2
                combinedData.put("order_details", index2Lookup.getOrDefault(orderId, null));
                // Add matching data from index_3
                combinedData.put("product_list", index3Lookup.getOrDefault(orderId, new ArrayList<>()));
                finalCombinedResults.add(combinedData);
            }
        }

        return finalCombinedResults;
    }
}

If you need to run multiple independent queries across your indices (without strict dependencies between them), the MultiSearchRequest lets you send all queries in a single HTTP call—this is more efficient than making separate requests.

Example code:

import org.elasticsearch.action.search.MultiSearchRequest;
import org.elasticsearch.action.search.MultiSearchResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;

import java.io.IOException;

public class MultiSearchDemo {

    public void runMultiIndexSearch(RestHighLevelClient client) throws IOException {
        MultiSearchRequest multiRequest = new MultiSearchRequest();

        // Add query for index_1 (your original query)
        SearchRequest index1Req = new SearchRequest("index_1");
        index1Req.source(new SearchSourceBuilder().query(QueryBuilders.termQuery("user", "kimchy")));
        multiRequest.add(index1Req);

        // Add query for index_2 (e.g., all recent orders)
        SearchRequest index2Req = new SearchRequest("index_2");
        index2Req.source(new SearchSourceBuilder().query(QueryBuilders.rangeQuery("created_at").gte("now-7d")));
        multiRequest.add(index2Req);

        // Add query for index_3 (e.g., top 10 popular products)
        SearchRequest index3Req = new SearchRequest("index_3");
        index3Req.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()).size(10));
        multiRequest.add(index3Req);

        // Execute all queries in one go
        MultiSearchResponse multiResponse = client.msearch(multiRequest, RequestOptions.DEFAULT);

        // Process each response individually
        MultiSearchResponse.Item[] responses = multiResponse.getResponses();
        SearchResponse index1Resp = responses[0].getResponse();
        SearchResponse index2Resp = responses[1].getResponse();
        SearchResponse index3Resp = responses[2].getResponse();

        // Now you can handle each index's data and associate them as needed in your code
    }
}

If your use case allows it, the most efficient long-term solution is to denormalize your data at write time. Instead of keeping related data in separate indices, embed nested structures directly into the main document. For example, if an index_1 document needs related index_2 and index_3 data, add those nested objects to the index_1 document when you index it.

This aligns with Elasticsearch's design philosophy of optimizing read performance over write complexity—you'll get all the data you need in a single search query without any post-processing.

内容的提问来源于stack exchange，提问作者tisispa1