如何限制Magento Elasticsearch索引字段数量及解决相关报错

问题背景 #

我的Magento 2产品索引因站点包含上百个客户组+40多个国家/网站，Magento会为这些组合的任意搭配索引价格数据，导致索引字段膨胀至34000余个。目前在Kibana中查看索引频繁触发“limit exceeded”报错，同时担忧这种架构带来的性能损耗。

已尝试的方案 #

• 编写静态映射，将price_*字段设为index: false，但不确定如何应用到频繁重建的索引，还怕操作不当导致站点崩溃：

  "dynamic_templates": [
          {
            "price_mapping": {
              "match": "price_*",
              "match_mapping_type": "string",
              "mapping": {
                "index": false,
                "store": true,
                "type": "double"
              }
            }
          },
  

• 想修改Kibana查询避免自动展开[*]，但不清楚具体操作方法；
• 考虑设置index.mapping.total_fields.limit为超大值，但担心影响Kibana运行。

后续尝试遇到的报错 #

使用索引模板后，字段数降至684，但重建目录搜索索引时出现报错：

Rejecting mapping update to [rcb2b_product_1_v2] as the final mapping would have more than 1 type: [_doc, document]"}]

当前使用Elasticsearch 7.13.1，曾针对该报错寻找配置方案。

EDIT2：已解决上述类型冲突报错，但又出现新的映射冲突，索引再次被大量类似price_749_12格式的字段填满：

"price_749_12":{"type":"double","store":true},"price_749_7":{"type":"double","store":true},"price_750_12":{"type":"double","store":true},"price_750_7":{"type":"double","store":true},"price_751_12":{"type":"double","store":true},"price_751_7":{"type":"double","store":true},"price_752_12":{"type":"double","store":true},"price_752_7":{"type":"double","store":true},"price_753_12":{"type":"double","store":true},"price_753_7":{"type":"double","store":true},"price_754_12":{"type":"double","store":true},"price_754_7":{"type":"double","store":true},"price_755_12":{"type":"double","store":true},"price_755_7":{"type":"double","store":true},"price_756_12":{"type":"double","store":true},"price_756_7":{"type":"double","store":true},"price_759_12":{"type":"double","store":true},"price_759_7":{"type":"double","store":true},"price_760_12":{"type":"double","store":true},"price_760_7":{"type":"double","store":true},"price_761_12":{"type":"double","store":true},"price_761_7":{"type":"double","store":true},"price_763_12":{"type":"double","store":true},"price_763_7":{"type":"double","store":true},"price_764_12":{"type":"double","store":true},"price_764_7":{"type":"double","store":true},"price_765_12":{"type":"double","store":true},"price_765_7":{"type":"double","store":true},"price_766_12":{"type":"double","store":true},"price_766_7":{"type":"double","store":true},"price_767_12":{"type":"double","store":true},"price_767_7":{"type":"double","store":true}

目前暂时陷入困境，寻求可行的解决方案。

可行解决方案 #

1. 修正动态模板并确保索引模板优先级 #

• 原动态模板中match_mapping_type: "string"不符合实际情况（price字段为double类型），建议删除该参数或改为match_mapping_type: "double"，确保所有price_*字段都能匹配到规则；
• 创建索引模板时设置较高order值（比如100），确保优先级高于Magento自动生成的模板，示例模板：

  PUT _index_template/magento_product_price_template
  {
    "index_patterns": ["rcb2b_product_*"],
    "order": 100,
    "template": {
      "mappings": {
        "dynamic_templates": [
          {
            "price_fields": {
              "match": "price_*",
              "mapping": {
                "type": "double",
                "index": false,
                "store": true
              }
            }
          }
        ]
      }
    }
  }
  

• 重建索引前删除旧的冲突模板，保证新模板生效。

2. 从Magento根源减少价格字段生成 #

• 后台配置调整：进入Stores > Configuration > Catalog > Catalog Search，检查Elasticsearch相关设置中是否有控制价格字段索引的选项，限制仅生成当前站点/客户组所需的价格字段；
• 自定义模块拦截：重写Magento\CatalogSearch\Model\Indexer\Fulltext\Action\DataProvider类，过滤掉不需要的客户组+国家组合，减少冗余字段生成。

3. 调整Kibana避免字段自动展开 #

• 在Kibana的Index Patterns设置中，找到对应产品索引模式，编辑时取消勾选Allow auto-expand to new fields，阻止Kibana自动加载所有字段；
• 查询时手动指定需要的字段，比如用product_name, price_default代替通配符*。

4. 临时缓解：调整Elasticsearch字段限制 #

若上述方案暂时无法实施，可临时调整字段限制值（不建议长期使用）：

PUT rcb2b_product_1_v2/_settings
{
  "index.mapping.total_fields.limit": 40000
}

内容的提问来源于stack exchange，提问作者heliogabal