Elasticsearch Java嵌套字段聚合：Count正确但Sum子聚合异常排查

Elasticsearch索引结构如下（places为嵌套类型，包含MAIN_LOCATION和AFFILIATE两种类型）：

[
  {
    "_index" : "hotel",
    "_type" : "_doc",
    "_id" : "13171",
    "_score" : 6.072218,
    "_source" : {
      "_class" : "hotel",
      "id" : 13171,
      "places" : [{"type" : "MAIN_LOCATION","placeId" : 2032}],
      "numberOfRecommendations" : 0
    }
  },
  {
    "_index" : "hotel",
    "_type" : "_doc",
    "_id" : "7146",
    "_score" : 6.072218,
    "_source" : {
      "_class" : "hotel",
      "id" : 7146,
      "places" : [{"type" : "MAIN_LOCATION","placeId" : 2032}],
      "numberOfRecommendations" : 1
    }  
  },
  {
    "_index" : "hotel",
    "_type" : "_doc",
    "_id" : "7146",
    "_score" : 6.072218,
    "_source" : {
      "_class" : "hotel",
      "id" : 7146,
      "places" : [{"type" : "AFFILIATE","placeId" : 2032}],
      "numberOfRecommendations" : 3
    }  
  }
]

需求：统计特定地点的酒店数量，以及MAIN_LOCATION类型下的推荐总数（示例中应返回酒店数2、推荐数1）。

现有Java代码能正确统计酒店数量，但推荐数总和始终为0，生成的查询及输出如下：

生成的ES查询 #

{
  "query": {
    "bool" : {
      "must" : [
        {
          "nested" : {
            "query" : {
              "bool" : {
                "must" : [
                  {"term" : {"places.type" : {"value" : "MAIN_LOCATION","boost" : 1.0}}},
                  {"terms" : {"places.placeId" : [7146],"boost" : 1.0}}
                ],
                "adjust_pure_negative" : true,
                "boost" : 1.0
              }
            },
            "path" : "places",
            "ignore_unmapped" : false,
            "score_mode" : "min",
            "boost" : 1.0
          }
        },
        {
          "nested" : {
            "query" : {"exists" : {"field" : "places","boost" : 1.0}},
            "path" : "places",
            "ignore_unmapped" : false,
            "score_mode" : "none",
            "boost" : 1.0
          }
        }
      ],
      "adjust_pure_negative" : true,
      "boost" : 1.0
    }
  },
  "aggs": {
    "nestedPlaces":{
      "nested":{"path":"places"},
      "aggregations":{
        "placeFilter":{
          "filters":{
            "filters":[{
              "bool":{
                "must":[
                  {"term":{"places.type":{"value":"MAIN_LOCATION","boost":1.0}}},
                  {"terms":{"places.placeId":[7146],"boost":1.0}}
                ],
                "adjust_pure_negative":true,
                "boost":1.0
              }
            }],
            "other_bucket":false,
            "other_bucket_key":"_other_"
          },
          "aggregations":{
            "group_by_place_id":{
              "terms":{
                "field":"places.placeId",
                "size":193,
                "min_doc_count":1,
                "shard_min_doc_count":0,
                "show_term_doc_count_error":false,
                "order":[{"_count":"desc"},{"_key":"asc"}],
                "include":["7146"]
              },
              "aggregations":{
                "totalRecommendationsForPlace":{
                  "sum":{"field":"numberOfRecommendations"}
                }
              }
            }
          }
        }
      }
    }
  }
}

当前查询输出 #

"aggregations" : {
  "nestedPlaces" : {
    "doc_count" : 7,
    "placeFilter" : {
      "buckets" : [
        {
          "doc_count" : 3,
          "group_by_place_id" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : 2032,
                "doc_count" : 3,
                "totalRecommendationsForPlace" : {"value" : 0.0}
              }
            ]
          }
        }
      ]
    }
  }
}

问题出在聚合上下文的范围：

• 使用nested聚合后，聚合进入了places嵌套文档的上下文，此时统计的是每个嵌套的places元素，而非根文档。
• numberOfRecommendations是根文档字段，在嵌套上下文里无法直接访问，因此sum聚合无法获取有效值，结果始终为0。

需要使用**reverse_nested聚合**跳出嵌套上下文，回到根文档层面，再对numberOfRecommendations做sum统计。

修改后的Java代码 #

调整聚合部分，在terms聚合的子聚合中先添加reverse_nested，再在其内部做sum：

// 原terms聚合部分修改
TermsAggregationBuilder aggregationBuilders =
    AggregationBuilders.terms(aggregationGroupByPlaceId)
        .field("places.placeId")
        .size(filter.getPlaceIds().size())
        .includeExclude(new IncludeExclude(includedPlaceIds, null))
        // 添加reverse_nested回到根文档上下文
        .subAggregation(AggregationBuilders.reverseNested("to_root")
            // 在根文档层面统计推荐数总和
            .subAggregation(AggregationBuilders.sum("totalRecommendationsForPlace")
                .field("numberOfRecommendations")));

修改后的ES查询结构 #

对应的聚合部分会变成：

"group_by_place_id":{
  "terms":{
    "field":"places.placeId",
    "size":193,
    "min_doc_count":1,
    "shard_min_doc_count":0,
    "show_term_doc_count_error":false,
    "order":[{"_count":"desc"},{"_key":"asc"}],
    "include":["7146"]
  },
  "aggregations":{
    "to_root":{
      "reverse_nested":{},
      "aggregations":{
        "totalRecommendationsForPlace":{
          "sum":{"field":"numberOfRecommendations"}
        }
      }
    }
  }
}

结果解析逻辑调整 #

同时需要修改getTotalRecommendationsForPlace方法，先获取reverse_nested聚合，再从中取sum值：

private int getTotalRecommendationsForPlace(Terms.Bucket bucket) {
  var reverseNestedAgg = bucket.getAggregations().get("to_root");
  if (reverseNestedAgg != null) {
    var aggregationTotalRecommendation = reverseNestedAgg.getAggregations().get("totalRecommendationsForPlace");
    if (aggregationTotalRecommendation != null) {
      return (int) ((ParsedSum) aggregationTotalRecommendation).getValue();
    }
  }
  return 0;
}

修改后，聚合会正确回到根文档统计numberOfRecommendations的总和，示例场景中会返回：

• 酒店数：2
• 推荐数：1

内容的提问来源于stack exchange，提问作者Sandeep Nair