Elasticsearch数据流中必填@timestamp字段的作用及ILM迁移场景下的相关疑问

Great question! It’s totally reasonable to wonder why @timestamp is required when writes already go to the current backing index. Let’s break down the key reasons this field is non-negotiable for data streams and Index Lifecycle Management (ILM):

• ILM can’t automate lifecycle actions without it
  ILM policies rely on @timestamp to trigger phase transitions (like moving data from hot to warm storage, or deleting old data). For example, if your policy says "archive data after 90 days", Elasticsearch uses the oldest @timestamp values in each backing index to determine when that threshold is hit. Without this timestamp, there’s no way for ILM to know when data was generated or ingested—rendering automated lifecycle management useless.

• Time-based index pruning supercharges query performance
  When you query a data stream, Elasticsearch can use @timestamp to skip entire backing indices that don’t contain data in your target time range. This "index pruning" cuts down on the amount of data that needs to be scanned, making queries way faster—especially as your data stream grows to include months or years of historical data. Without @timestamp, Elasticsearch has to check every backing index every time, which slows things down significantly.

• Data streams are built for time-series data
  Data streams aren’t just a wrapper for multiple indices—they’re purpose-built for time-series use cases (like logs, metrics, or sensor data). The @timestamp field is the anchor for all the optimizations that make data streams efficient: from how indices are rolled over (e.g., daily or monthly indices based on data age) to how shards are allocated and merged. Without it, you’re not leveraging the core design of data streams.

• Consistent data organization
  Even though writes go to the current index, @timestamp ensures that historical data is logically grouped by time across backing indices. This makes it easier to manage, archive, or delete old data in bulk. For example, if you roll over indices weekly, each index will contain data from a specific week—all aligned using the @timestamp field.

At the end of the day, @timestamp isn’t just a required field—it’s the backbone of how data streams and ILM work together to make time-series data management scalable and efficient.

内容的提问来源于stack exchange，提问作者Rahul