Elasticsearch v7.0嵌套数组查询报错排查及需求实现求助

我使用Elasticsearch v7.0，现有索引包含nested类型的child字段，该字段有db_name和con_full_name属性。需要筛选出主文档的db_name与至少一个子文档db_name相同的文档。

尝试用Painless脚本循环child字段实现逻辑时触发了"all shards failed"错误，需要排查原因并提供正确查询方案。

索引映射 #

PUT /test_dup_contacts_new
{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "properties": {
      "con_full_name": { "type": "text" },
      "db_name": { "type": "text" },
      "child": { 
            "type": "nested",
            "properties":{
                "con_full_name": { "type": "text" },
                "db_name": { "type": "text" }
            } 
        }
    }
  }
}

文档示例 #

[
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "3HwmhIMBS7IAsS38K8vg",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "Jesse Grinstead SirManBoy",
          "db_name" : "bc483ac37173a45849add211047a6dd7",
          "child" : [
            {
              "con_full_name" : "Jesse child 1",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            },
            {
              "con_full_name" : "Jesse child 2",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            }
          ]
        }
      },
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "3XwmhIMBS7IAsS388MtR",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "Jesse Grinstead",
          "db_name" : "bc483ac37173a45849add211047a6dd7",
          "child" : [
            {
              "con_full_name" : "Grinstead child 1",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            },
            {
              "con_full_name" : "Grinstead child 2",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            }
          ]
        }
      },
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "3nw9hIMBS7IAsS38LMsf",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "Test Dev",
          "db_name" : "bc483ac37173a45849add211047a6dd7",
          "child" : [
            {
              "con_full_name" : "Grinstead child 1",
              "db_name" : "test"
            },
            {
              "con_full_name" : "Grinstead child 2",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            }
          ]
        }
      },
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "33xHhIMBS7IAsS386ctL",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "Test Dev",
          "db_name" : "bc483ac37173a45849add211047a6dd7",
          "child" : [
            {
              "con_full_name" : "Grinstead child 1",
              "db_name" : "test"
            }
          ]
        }
      },
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "4HxjiYMBS7IAsS38w8sU",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "devid miller",
          "db_name" : "bc483ac37173a45849add211047a6dd7",
          "child" : [
            {
              "con_full_name" : "devid child 1",
              "db_name" : "test_db"
            },
            {
              "con_full_name" : "devid child 2",
              "db_name" : "bc483ac37173a45849add211047a6dd7"
            },
            {
              "con_full_name" : "devid child 3",
              "db_name" : "test_db"
            },
            {
              "con_full_name" : "devid child 4",
              "db_name" : "test_db"
            }
          ]
        }
      },
      {
        "_index" : "test_dup_contacts_new",
        "_type" : "_doc",
        "_id" : "43xliYMBS7IAsS38KMsW",
        "_score" : 1.0,
        "_source" : {
          "con_full_name" : "Snu dev",
          "db_name" : "test_db",
          "child" : [
            {
              "con_full_name" : "Snu child 1",
              "db_name" : "test_db"
            }
          ]
        }
      }
    ]

失败的查询语句 #

GET test_dup_contacts_new/_search
{
  "query": {
    "bool": {
      "filter":{
      "script":{
      "script": {
        "lang": "painless",
        "source": "def num = 0; for (int i=0; i<params._source['child'].length; i++){num++;} if(num>0){ return true;} else{ return false;}"
        
      }
    }
    }
    } 
    
  }
}

错误信息 #

"error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "i=0; i<params._source['child'].length; i++){",
          "             ^---- HERE"
        ],
        "script": "def num = 0; for (int i=0; i<params._source['child'].length; i++){num++;} if(num>0){ return true;} else{ return false;}",
        "lang": "painless"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
}

错误原因分析 #

• params._source访问不可靠：过滤脚本中直接访问params._source不被推荐，Elasticsearch默认不会在脚本中加载完整文档源，嵌套类型字段无法通过这种方式正确获取结构。
• 未处理空值场景：若文档中child字段不存在或为空数组，访问params._source['child'].length会触发空指针异常，导致分片查询失败。
• 脚本逻辑偏离需求：当前脚本仅判断child数组是否非空，未实现主文档与子文档db_name匹配的核心逻辑。

正确查询方案 #

方案1：Nested Query + Script Query（推荐） #

利用nested查询的上下文，直接在脚本中对比主文档和子文档的db_name值：

GET test_dup_contacts_new/_search
{
  "query": {
    "nested": {
      "path": "child",
      "query": {
        "script": {
          "script": {
            "lang": "painless",
            "source": "doc['db_name'].value == doc['child.db_name'].value"
          }
        }
      },
      "inner_hits": {} // 可选，返回匹配的子文档详情
    }
  }
}

方案2：改进后的Painless脚本（适合小数据量场景） #

若必须使用脚本过滤，需正确访问字段并处理空值情况：

GET test_dup_contacts_new/_search
{
  "query": {
    "bool": {
      "filter": {
        "script": {
          "lang": "painless",
          "source": """
            String parentDbName = doc['db_name'].value;
            if (doc.containsKey('child.db_name')) {
              for (String childDbName : doc['child.db_name']) {
                if (childDbName.equals(parentDbName)) {
                  return true;
                }
              }
            }
            return false;
          """
        }
      }
    }
  }
}

注意：此方式性能低于nested query，需遍历所有子文档字段值，仅适合数据量较小的场景。

内容的提问来源于stack exchange，提问作者Shailesh Jadav