AWS Elasticsearch Service告警触发后如何获取完整变量值？

Let me walk through why most of your alert variables are coming up empty and how to fix this—this is a common gotcha with Open Distro's alerting system!

1. Verify Your Query Returns Hits During the Monitor's Execution Window #

Even if your query works when you run it manually, the monitor might be executing at a time when there are no non-200 requests in the now-1m window. Open Distro’s alerting ties the query’s relative time range to the monitor’s scheduled run time.

• Check your monitor’s execution history to see exactly when it runs, then execute your query in Kibana Dev Tools at that exact moment to confirm it returns hits.
• If your monitor’s interval is longer than 1 minute (e.g., every 5 minutes), your now-1m range might miss the window where non-200 requests occurred. Adjust the query’s time range to match the monitor’s interval (e.g., now-5m for a 5-minute interval).

2. Fix Your Condition Configuration #

If your query returns hits but ctx.results is still empty, your condition might be set in a way that doesn’t capture those results:

• For "Number of results" conditions: Ensure your threshold aligns with expected results (e.g., set to "greater than 0" to alert whenever non-200 requests exist). If the threshold is too high, the alert might still fire if you’ve enabled "alert on no results," but ctx.results will stay empty.
• For "Custom condition" (Painless): Double-check your script is correctly referencing query results and returning true when you want the alert to trigger. A poorly written script might fire the alert but fail to pass through results to the ctx object.

3. Correct Variable Syntax in Your Slack Template #

Many users miss the nested properties in the ctx object, leading to seemingly empty values. Here’s how to properly access the data you need:

• To get the total number of hits: Use {{ctx.results.0.hits.total.value}} (note the .value suffix—total is an object with a nested value field).
• To loop through sample hits: You need to iterate over ctx.results.0.hits.hits and reference _source fields.

Example working Slack template:

🚨 Alert: {{ctx.trigger.name}}
⏱️ Period: {{ctx.periodStart}} → {{ctx.periodEnd}}
🔢 Total non-200 requests: {{ctx.results.0.hits.total.value}}

📋 Sample Requests:
{{#each ctx.results.0.hits.hits}}
- Status: {{_source.status}} | Time: {{_source.@timestamp}} | URL: {{_source.url}}
{{/each}}

4. Debug Query Execution Issues #

If the above steps don’t resolve the issue, check if the query is failing silently (even if the alert triggers). To debug:

• Enable debug logs for the alerting plugin by adding this to your Elasticsearch config (AWS users can set this via the console’s advanced settings):

  logger.org.opendistroforelasticsearch.alerting: DEBUG
  

• Check Elasticsearch logs for entries related to your monitor—look for errors in query execution or condition evaluation that might be hiding results.

5. Confirm Monitor Permissions #

Ensure the role associated with your alerting monitor has read access to the index you’re querying. If the monitor can’t read the index, the query will return no results, leading to empty variables. Verify this by running the query as the alerting role in Kibana Dev Tools.

内容的提问来源于stack exchange，提问作者antken