如何用Lucene查询Kibana ElasticSearch中符合条件的JSON格式message字段

Got it, let's tackle this query problem for your Elasticsearch Kibana dashboard. Since your message field is stored as a JSON string, we have two main approaches to filter for orders > 30 and version > 3.4:

1. Quick Fix: Script Query (No Mapping Changes) #

If you can't modify your index mapping right now, you can use a Lucene script query to parse the JSON string on the fly and apply your conditions. Here's the query you can use in Kibana's search bar or Dev Tools:

{
  "query": {
    "bool": {
      "must": [
        {
          "script": {
            "script": """
              def jsonMsg = org.json.JSONObject.parse(params._source.message);
              return jsonMsg.orders > 30 && Float.parseFloat(jsonMsg.version) > 3.4;
            """
          }
        }
      ]
    }
  }
}

Key Notes: #

• This script pulls the raw message string from the document source, parses it into a JSON object, then checks both conditions.
• Make sure your Elasticsearch instance allows script execution (adjust script.allowed_types and related settings in elasticsearch.yml if needed, especially for newer ES versions).
• Performance-wise, this works for small datasets, but isn't ideal for large volumes since parsing JSON on every query adds overhead.

2. Better Long-Term: Structured Indexing (Recommended) #

To make queries faster and cleaner, you should configure your setup to index the JSON content in message as structured fields instead of a single string. Here's how:

Step 1: Adjust Your Logback Appender #

Instead of converting the JSON object to a string and sending it as the message field, configure your logback-elasticsearch-appender to send the JSON structure directly. This way, Elasticsearch will automatically index version, model, and orders as nested fields under message.

Update your logging code to avoid converting the JSON to a string:

// Instead of this:
// log.info(new org.json.JSONObject(arg).toString());

// Send the JSON object directly (match your appender's expected input format)
log.info(arg);

Step 2: Update Index Mapping (If Needed) #

If your index already exists, update its mapping to recognize message as an object type:

PUT /your-index-name/_mapping
{
  "properties": {
    "message": {
      "type": "object",
      "enabled": true
    }
  }
}

Step 3: Simple Lucene Query #

Once the fields are indexed properly, you can use a straightforward query in Kibana:

message.orders:>30 AND message.version:>"3.4"

Or as a structured JSON query (great for more complex filters):

{
  "query": {
    "bool": {
      "must": [
        { "range": { "message.orders": { "gt": 30 } } },
        { "range": { "message.version": { "gt": "3.4" } } }
      ]
    }
  }
}

Key Notes: #

• Using structured fields is way more efficient because Elasticsearch indexes each nested field directly, no runtime parsing required.
• The version field is treated as a string here, but since semantic version strings sort logically (e.g., "3.4.2" > "3.4"), the range query works perfectly for your case.

内容的提问来源于stack exchange，提问作者Achaius