Elasticsearch：节点内多数据路径的分片分配与迁移机制咨询

Elasticsearch 7.6: Multi-Data Path Behavior Breakdown #

Great question—this is a tricky area that’s often overlooked in cluster docs. Let’s break down each of your questions clearly:

1. When & How Does ES Decide to Create/Migrate Shards Across Node Data Paths? #

First, let’s start with the basics: when you configure multiple path.data directories (comma-separated in elasticsearch.yml), Elasticsearch handles shard placement and movement like this:

• Shard Creation: By default, ES uses a round-robin approach across available data paths, but it’s not blind—it prioritizes paths with lower current disk usage to keep balance across the node. It also avoids placing new shards on paths that are already approaching disk watermark thresholds.
• Intra-Node Shard Migration: Out of the box, Elasticsearch does not automatically migrate existing shards between data paths on the same node. Automatic migrations are focused on cross-node balancing (to spread load across the cluster). If you need to move a shard from one path to another on the same node, you’ll have to do it manually using the _cluster/reroute API or adjust cluster settings to trigger a relocation.

2. Do Disk Watermarks Trigger Intra-Node Shard Migrations? #

Short answer: No, not by default. The disk watermark thresholds (cluster.routing.allocation.disk.watermark.low, high, flood_stage) only control cross-node shard allocation:

• When a node hits the high watermark, ES stops allocating new shards to it and starts moving existing shards to other nodes with available space.
• When it hits flood_stage, ES will even relocate primary shards off the node to prevent disk exhaustion.

These watermarks don’t trigger moving shards between different data paths on the same node. If one path on a node is full but others are empty, ES will just stop placing new shards on the full path—existing shards on that path stay put unless you move them manually.

3. Can You Specify a Data Path for New Indexes? #

Absolutely! You can define a specific data path for an index at creation time using the index.data_path setting. This path must be one of the directories listed in the node’s path.data configuration (otherwise ES will throw an error).

For example, to create an index that uses /mnt/data2 (assuming this is in your path.data list):

PUT /my_custom_path_index
{
  "settings": {
    "index.data_path": "/mnt/data2"
  }
}

You can also set this in an index template if you want all new indexes matching the template to use a specific path.

4. Official Documentation References #

The key sections in the Elasticsearch 7.6 docs that cover this behavior are:

• Path Settings: Covers configuring multiple path.data directories and basic allocation logic.
• Disk-Based Shard Allocation: Details how disk watermarks affect cross-node shard movement.
• Index Settings: Explains the index.data_path parameter for per-index path assignment.

内容的提问来源于stack exchange，提问作者Flatline1963