能否手动构建Elasticsearch scroll_id以离线生成下一页请求URL？

Short answer: No, you can't manually construct a valid Elasticsearch scroll_id on your own—here's why, plus better alternatives that fit your offline workflow:

Why Building a scroll_id Isn't Feasible #

Elasticsearch's scroll_id isn't a simple, predictable value you can generate client-side. It's a server-encoded string packed with critical context about your scroll query, including:

• A snapshot of the cluster state and index data at the time of the initial scroll request (scroll queries rely on this point-in-time snapshot to avoid consistency issues as data changes)
• The current cursor position across all index shards
• Metadata about the original query, routing rules, and shard allocation

This data is encoded (and often signed, depending on your ES version) to prevent tampering and ensure the server can correctly resume the scroll. Without access to the cluster's internal state and encoding logic, you have no way to generate a valid scroll_id that Elasticsearch will accept.

Better Alternatives for Offline Pagination #

Since you want to avoid relying on a live scroll_id from the initial request, these two approaches work better for offline-generated requests:

1. Use search_after Pagination #

This is the recommended approach for scalable, stateless pagination that you can build offline. Here's how it works:

• First, run your initial query with size=20 (your max per-page limit) and specify a unique, sortable field (like _id, or a business-specific unique timestamp/ID) as part of your sort criteria. For example:

  GET /your_index/_search
  {
    "size": 20,
    "sort": [
      {"timestamp": "asc"},
      {"_id": "asc"} // Ensures uniqueness even if timestamps overlap
    ]
  }
  

• Take the sort values from the last document in the response (e.g., ["2024-05-20T12:00:00Z", "abc123"]).
• To build your next page request offline, add the search_after parameter with those values:

  GET /your_index/_search
  {
    "size": 20,
    "sort": [
      {"timestamp": "asc"},
      {"_id": "asc"}
    ],
    "search_after": ["2024-05-20T12:00:00Z", "abc123"]
  }
  

• Repeat this process, updating search_after with the last document's sort values each time, until you get fewer than 20 results (indicating you've reached the end).

Pros: No server-side state to track, works offline once you have the last page's sort values, and avoids the performance issues of deep pagination.

2. Use from/size Pagination (For Small Datasets) #

If your total result set is small (under 10,000 documents, Elasticsearch's default limit), you can use the from parameter to offset your results:

• Initial request: from=0&size=20
• Next page: from=20&size=20
• And so on: from=40&size=20, etc.

Warning: This approach doesn't scale well for large datasets. Elasticsearch has to fetch from + size documents from every shard, sort them, and then truncate to the size you requested. For from values over 1000, performance drops off significantly, and you'll hit the index.max_result_window limit by default.

Final Notes #

Scroll IDs are inherently tied to the server's state when the scroll is initiated, so manual construction is impossible. Stick with search_after for a reliable, offline-friendly way to paginate through your Elasticsearch results—it's designed exactly for use cases where you don't want to rely on maintaining a server-side cursor like scroll_id.

内容的提问来源于stack exchange，提问作者user3285241