ElasticSearch聚合[unique_users]定义缺失报错及用户列表需求处理

Hey there! Let's break down how to fix your Elasticsearch aggregation issues in Ruby on Rails step by step.

First, let's unpack the problems you're facing #

1. Your initial aggregation only returned a user count, no email list
2. After modifying code, you got both count and emails but had an unwanted doc_count in the response
3. Removing the filter: { match_all: {} } threw a "Missing definition for aggregation [unique_users]" error

Fixing the aggregation error & removing doc_count #

Here's what's going wrong and how to fix it:

1. Why the "Missing definition" error happens

When you remove the match_all filter, you might have left your query structure incomplete. Elasticsearch expects either a query (even an empty one) or to explicitly tell it you don't need to return any matching documents. The fix here is to set size: 0 (we don't need individual user docs, just aggregation results) and define your aggregations directly at the top level of the search request.

2. Getting both unique count and email list

To get both the total unique user count and their email list, combine two aggregations:

• A terms aggregation to pull all unique email addresses (use the keyword version of your email field to avoid splitting emails into tokens)
• A cardinality aggregation to calculate the total number of unique users (this is more efficient than counting buckets manually for large datasets)

3. Removing the unwanted doc_count

Setting size: 0 tells Elasticsearch not to return any matching documents, so the root-level doc_count (from hits) disappears entirely. If you're referring to the doc_count inside each email bucket (which counts how many times that email appears), you can ignore it or extract just the email keys in your Ruby code.

Working Ruby on Rails Code Example #

# Run the optimized aggregation query
search_results = User.search(
  size: 0, # Skip returning individual docs to eliminate root doc_count
  aggregations: {
    unique_emails: {
      terms: {
        field: 'email.keyword', # Critical: use keyword field for exact unique matches
        size: 10000 # Adjust this to be larger than your total unique user count to avoid truncation
      }
    },
    total_unique_users: {
      cardinality: {
        field: 'email.keyword'
      }
    }
  }
)

# Extract the data you need
unique_email_list = search_results.aggregations['unique_emails']['buckets'].map { |bucket| bucket['key'] }
total_unique_users = search_results.aggregations['total_unique_users']['value']

# Use the results however you need
puts "Total unique users: #{total_unique_users}"
puts "Unique emails: #{unique_email_list.join(', ')}"

Quick Notes #

• Make sure your Elasticsearch mapping for the email field includes a keyword sub-field (most default text mappings have this automatically, but double-check if you get unexpected results)
• Adjust the size parameter in the terms aggregation to be larger than your maximum expected unique user count—Elasticsearch defaults to returning only 10 buckets otherwise

内容的提问来源于stack exchange，提问作者Ravindra Yadav