Elasticsearch中time字段映射至@timestamp字段的问题与解决

源数据格式 #

我的Elasticsearch索引源数据格式如下：

"_source": {
    "email": "smithamber@example.com",
    "time": "2022-09-08T13:52:50.347861",
    "message": "Pattern thank talk mention. Manage nearly tell beat. Difficult husband feel talk radio however.",
    "sIp": "192.168.11.156",
    "dIp": "80.254.211.60",
    "ts": "2022-09-08T13:52:50"
}

无效的runtime_mappings配置 #

我希望将@timestamp字段动态映射为源字段time，最初使用了以下runtime_mappings配置：

"runtime_mappings": {
    "@timestamp": {
        "type": "date",
        "format": "yyyyMMdd'T'HHmmss.SSSZ",
        "script": {
            "source": "if (doc[\"time\"].size() == 0) {return} else {return doc[\"time\"].value;}",
            "lang": "painless"
         }
        }
    }

查询结果异常 #

执行以下range查询@timestamp字段时，无命中结果：

{
    "query": {
        "range": {
            "@timestamp": {
                "gte": "now-5d",
                "lte": "now"
            }
        }
    }
}

查询返回结果：

{
"took": 20,
"timed_out": false,
"_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
},
"hits": {
    "total": {
        "value": 0,
        "relation": "eq"
    },
    "max_score": null,
    "hits": []
}
}

但使用相同条件查询time字段时，能正常获取过滤后的所有文档：

{
"took": 27,
"timed_out": false,
"_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
},
"hits": {
    "total": {
        "value": 10000,
        "relation": "gte"
    },
    "max_score": 1.0,
    "hits": [
        {
            "_index": "topic-indexer-xxx",
            "_id": "c28sIYMB0xJUJru8c47O",
            "_score": 1.0,
            "_source": {
                "email": "albertthompson@example.com",
                "time": "2022-09-07T15:25:33.672016",
                "message": "Candidate future staff ever former run. Like quality personal specific trouble cell money move. Available majority memory model thing TV wrong. Summer anyone light key.",
                "sIp": "192.168.103.75",
                "dIp": "191.27.68.163"
            }
        },
        ....
    ]
}
}

无效的dynamic_templates配置 #

我也尝试了dynamic_templates配置，但查询@timestamp字段仍无结果：

{
"dynamic_templates": [
  {
    "@timestamp": {
      "match":   "time",
      "mapping": {
        "type":       "date",
        "format": "strict_date_optional_time",
        "copy_to":    "@timestamp"
      }
    }
  }
]
}

最终生效的配置 #

参考@paulo的回复后，微调配置解决了问题，以下映射配置可正常工作，且能对@timestamp字段执行range查询：

{
  "runtime": {
    "@timestamp": {
      "type": "date",
      "script": {
        "source": "if (doc['time'].size() != 0){ emit(doc['time'].value.toEpochMilli());}",
        "lang": "painless"
      }
    }
  },
  "properties": {
      "@timestamp": {
          "type": "date"
      }
  }
}

内容的提问来源于stack exchange，提问作者Ankita Mehta