关于ElasticSearch中JSON文档倒排索引及等价SQL查询处理的咨询

Great question—this is a core concept that trips up a lot of folks coming from SQL to Elasticsearch, so let's unpack it clearly.

1. How Elasticsearch Builds Inverted Indexes for JSON Docs #

Unlike classic inverted indexes for unstructured text, Elasticsearch treats each JSON field as a separate "dimension" and builds specialized indexes for each field individually. Here's a breakdown:

• Per-field indexing: Every key in your JSON document gets its own inverted index (or index structures, depending on the field type). For example, if you have a document like:

  {
    "name": "John Doe",
    "age": 32,
    "bio": "John works as a software engineer in NYC."
  }
  

  Elasticsearch will create distinct indexes for name, age, and bio.

• Field type determines indexing behavior:

  • Text fields: Values are split into individual terms (via tokenization), standardized (lowercased, stripped of punctuation, etc.). For the name field (if mapped as text), "John Doe" becomes two terms: john and doe. Each term maps to the list of document IDs where it appears, plus metadata like term frequency and position (for phrase searches).
  • Keyword fields: Values are stored as exact, unmodified strings. If name was mapped as keyword, the entire "John Doe" is one term—only exact matches will hit this index.
  • Numeric/datetime fields: These use specialized index structures (like B-trees) optimized for range queries, but still tie back to document IDs.

• Term-to-document mapping: At its core, each field's inverted index is a dictionary where keys are terms (or numeric values) and values are pointers to the documents containing that term. For example, the term john in the name index would point to the ID of the sample document above.

2. What Happens When You Run a "WHERE name = john" Equivalent Query #

Let's say you run a query like this in Elasticsearch (the equivalent of your SQL example):

{
  "query": {
    "term": {
      "name": "john"
    }
  }
}

Or if name is a text field, you might use a match query (which handles analysis automatically):

{
  "query": {
    "match": {
      "name": "john"
    }
  }
}

Here's the step-by-step processing:

• Step 1: Query Parsing: Elasticsearch parses your query to identify the target field (name) and the search value (john). It checks the field's mapping to know how to handle the query.
• Step 2: Query Analysis (for text fields): If name is a text field, Elasticsearch runs the search value through the same analyzer used during indexing. So "John" (with a capital J) would get lowercased to john to match the indexed terms. For keyword fields, no analysis is done—we look for the exact string.
• Step 3: Look Up the Inverted Index: Elasticsearch goes to the name field's inverted index and finds all document IDs associated with the term john.
• Step 4: Scoring (optional): If you're using a match query (or any query that cares about relevance), Elasticsearch calculates a relevance score for each matching document (using algorithms like TF-IDF or BM25). If you use a filter instead of a query, this step is skipped—filters don't score results and are cached for faster future queries.
• Step 5: Retrieve Documents: Using the matched document IDs, Elasticsearch fetches the full JSON documents from the underlying storage (or cache if they're already in memory).
• Step 6: Sort & Return: By default, results are sorted by relevance score (highest first). You can override this with custom sort rules. Finally, the formatted results are sent back to the client.

A quick note: If you wanted an exact match on a text field (like your SQL =), you'd use a term query with the analyzed term (or map the field as keyword to avoid analysis entirely).

内容的提问来源于stack exchange，提问作者Calvin_Z