elasticsearch-java CreateIndexRequest调用withJson报属性缺失异常

问题现象 #

在SpringBoot环境下从已废弃的HLRC客户端迁移至新版elasticsearch-java 7.17.4客户端时，创建索引流程中使用CreateIndexRequest的withJson方法加载JSON格式的索引配置源，抛出异常提示存在缺失的必填属性。
核心疑问：

• JSON配置文件中是否必须包含所有属性？
• 为何同一份JSON配置在Kibana中仅传入所需属性即可正常运行，在已废弃的HLRC客户端中调用CreateIndexRequest也可正常生效，仅在新版Java客户端中报错？

环境依赖版本 #

• org.elasticsearch: elasticsearch 7.17.4
• co.elastic.clients: elasticsearch-java 7.17.4
• jakarta.json:jakarta.json-api 2.0.1
• com.fasterxml.jackson: jackson-databind 2.13.3

异常栈信息 #

co.elastic.clients.json.JsonpMappingException: Error deserializing 
co.elastic.clients.elasticsearch._types.analysis.TokenizerDefinition: 
co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 
'PathHierarchyTokenizer.bufferSize' (JSON path: 
settings.analysis.tokenizer.unix_path_tokenizer) (line no=15, column no=10, offset=377)

业务实现代码 #

final String assetJsonSource = "./config/elasticsearch/my_index_settings.json";
try (InputStream input = new FileInputStream(assetJsonSource)) {
  CreateIndexRequest request =
      CreateIndexRequest.of(builder -> builder.index(indexName).withJson(input));
  CreateIndexResponse response = client2.indices().create(request);
  boolean ack = Boolean.TRUE.equals(response.acknowledged());
} catch (IOException e) {
  log.error("Failed to create an index", e);
}

索引配置文件 #

{
  "settings": {
    "number_of_shards": 5,
    "max_ngram_diff": 2,
    "analysis": {
      "tokenizer": {
        "ngram_tokenizer": {
          "type": "ngram",
          "min_gram": "1",
          "max_gram": "3",
          "token_chars": ["letter", "digit", "punctuation", "symbol"]
        },
        "unix_path_tokenizer": {
          "type": "path_hierarchy"
        },
        "whitespace_tokenizer": {
          "type": "whitespace"
        },
        "keyword_tokenizer": {
          "type": "keyword"
        }
      },
      "analyzer": {
        "ngram_analyzer": {
          "tokenizer": "ngram_tokenizer",
          "char_filter": ["icu_normalizer"],
          "filter": ["lowercase"]
        },
        "lowercase_analyzer": {
          "tokenizer": "keyword",
          "filter": ["lowercase"]
        },
        "directory_path_analyzer": {
          "tokenizer": "unix_path_tokenizer"
        },
        "whitespace_analyzer": {
          "tokenizer": "whitespace_tokenizer"
        },
        "keyword_analyzer": {
          "tokenizer": "keyword_tokenizer"
        }
      },
      "normalizer": {
        "lowercase_normalizer": {
          "filter": ["lowercase"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "analyzer": "ngram_analyzer",
        "fields": {
          "lowercase": {
            "type": "keyword",
            "normalizer": "lowercase_normalizer"
          }
        }
      },
      "path": {
        "type": "text",
        "analyzer": "directory_path_analyzer",
        "fields": {
          "full": {
            "type": "keyword"
          }
        }
      },
      "originalSize": {
        "type": "double",
        "store": "true"
      },
      "assetCategory": {
        "type": "text",
        "analyzer": "keyword_analyzer",
        "search_analyzer": "whitespace_analyzer",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "mimetype": {
        "type": "keyword"
      },
      "importedBy": {
        "type": "integer",
        "store": "true"
      },
      "updatedBy": {
        "type": "keyword"
      },
      "importedAt": {
        "type": "date",
        "store": "true"
      },
      "updatedAt": {
        "type": "date",
        "store": "true"
      },
      "fileCreatedAt": {
        "type": "date",
        "store": "true"
      },
      "fileUpdatedAt": {
        "type": "date",
        "store": "true"
      },
      "metadataSet": {
        "type": "long"
      },
      "instanceId": {
        "type": "keyword"
      },
      "referenceId": {
        "type": "keyword"
      },
      "cutComment": {
        "type": "text",
        "analyzer": "ngram_analyzer",
        "fields": {
          "lowercase": {
            "type": "text",
            "analyzer": "lowercase_analyzer"
          }
        }
      },
      "comment": {
        "properties": {
          "userId": {
            "type": "long"
          },
          "value": {
            "type": "text",
            "analyzer": "ngram_analyzer",
            "fields": {
              "lowercase": {
                "type": "text",
                "analyzer": "lowercase_analyzer"
              }
            }
          },
          "updatedAt": {
            "type": "date"
          }
        }
      },
      "content": {
        "type": "text",
        "analyzer": "ngram_analyzer"
      },
      "shadow": {
        "type": "boolean",
        "store": "true"
      },
      "shadowUpdatedAt": {
        "type": "date",
        "store": "true"
      },
      "downloadValue": {
        "type": "long"
      },
      "collection": {
        "type": "long"
      },
      "sha1": {
        "type": "keyword"
      },
      "subtitle": {
        "type": "text",
        "analyzer": "ngram_analyzer"
      },
      "videoOcr": {
        "type": "text",
        "analyzer": "ngram_analyzer"
      },
      "version": {
        "type": "long"
      }
    },
    "dynamic_templates": [
      {
        "cmeta_str": {
          "match": "cmeta_str-*",
          "mapping": {
            "type": "text",
            "store": "true",
            "analyzer": "ngram_analyzer",
            "fields": {
              "lowercase": {
                "type": "keyword",
                "normalizer": "lowercase_normalizer"
              }
            }
          }
        }
      },
      {
        "cmeta_select": {
          "match": "cmeta_select-*",
          "mapping": {
            "type": "text",
            "store": "true",
            "analyzer": "ngram_analyzer",
            "fields": {
              "lowercase": {
                "type": "keyword",
                "normalizer": "lowercase_normalizer"
              }
            }
          }
        }
      },
      {
        "cmeta_bool": {
          "match": "cmeta_bool-*",
          "mapping": {
            "type": "boolean",
            "store": "true"
          }
        }
      },
      {
        "cmeta_double": {
          "match": "cmeta_double-*",
          "mapping": {
            "type": "double",
            "store": "true"
          }
        }
      },
      {
        "cmeta_date": {
          "match": "cmeta_date-*",
          "mapping": {
            "type": "date",
            "store": "true"
          }
        }
      },
      {
        "cmeta_multi_label": {
          "match": "cmeta_multi_label-*",
          "mapping": {
            "type": "long",
            "store": "true"
          }
        }
      }
    ]
  }
}

问题根因 #

报错和ES服务端无关，完全是新版Java客户端的客户端侧校验逻辑导致：

• Kibana、旧HLRC客户端的逻辑是直接将原始JSON请求透传给ES服务端，未传的可选参数由服务端自动填充默认值，不会在客户端做字段校验，所以不会报错。
• elasticsearch-java 7.17.x版本是强类型客户端，调用withJson时会先把传入的JSON完整反序列化为对应的Java实体类，而该版本的源码中错误地将PathHierarchyTokenizer等多个分词器、过滤器的带默认值字段（比如本次报错的bufferSize）标记为必填属性，反序列化时发现JSON中没有对应字段就直接抛出异常，请求根本不会发送到ES服务端。

解决方案 #

三种方案可按需选择：

• 补全JSON中缺失的默认值字段：针对报错的path_hierarchy分词器，补全所有官方默认参数即可，修改后的配置片段如下：

  "unix_path_tokenizer": {
    "type": "path_hierarchy",
    "delimiter": "/",
    "replacement": "/",
    "buffer_size": 1024,
    "reverse": false,
    "skip": 0
  }
  

  后续如果其他分词器、配置项报同类缺失属性错误，按照ES官方文档补全对应默认值即可。
• 绕过客户端强类型反序列化：不使用withJson方法加载全量配置，直接通过底层通用请求接口透传原始JSON，行为和旧HLRC、Kibana完全一致，不会触发客户端侧校验，示例代码：

  String indexConfig = Files.readString(Paths.get(assetJsonSource), StandardCharsets.UTF_8);
  // 调用底层low level client发原始请求
  Request putRequest = new Request("PUT", "/" + indexName);
  putRequest.setJsonEntity(indexConfig);
  client2.getLowLevelClient().performRequest(putRequest);
  

• 升级客户端版本：8.x版本的elasticsearch-java修复了大量这类“默认值字段被误标记为必填”的反序列化bug，但注意8.x客户端需要匹配8.x版本的ES服务端，跨大版本升级存在兼容性风险，7.x服务端不建议使用。

内容的提问来源于stack exchange，提问作者francissoria