Logstash Elasticsearch输出scripted_upsert脚本能否访问Event字段？

Great question! The short answer is: Yes, you can absolutely use Logstash event fields in your scripted_upsert script—you just need to pass them correctly into Elasticsearch's script context, which is where the upsert logic runs.

Why Your Initial Attempt Failed #

When you use scripted_upsert, the script executes inside Elasticsearch, not Logstash. That means Elasticsearch has no direct access to Logstash's event object. Your static value worked because it was a hardcoded string the Elasticsearch script engine could process directly, but trying to reference event (a Logstash-specific object) threw an invisible error (check your Logstash logs if you want to confirm this!) that prevented the upsert from completing.

The Correct Approach: Use Script Parameters #

To pass Logstash event fields into your upsert script, you need to use the script_params option in the Elasticsearch output plugin. This lets you inject dynamic values from the Logstash event into the Elasticsearch script as named parameters.

Here's a working example configuration:

output {
  elasticsearch {
    hosts => ["your-es-host:9200"]
    index => "your-target-index"
    document_id => "%{your-document-id-field}" # Required for update/upsert actions
    action => "update"
    scripted_upsert => true
    script => '
      # Initialize the source if the document doesn't exist yet (upsert case)
      if (ctx._source == null) {
        ctx._source = {};
      }
      # Assign the dynamic value from Logstash using the parameter
      ctx._source.name = params.event_name;
      # Add more field assignments here using params.your_field_name
    '
    script_params => {
      # Map Logstash event fields to script parameters
      "event_name" => "%{your-logstash-event-field}"
    }
  }
}

Key Details to Note: #

• document_id is mandatory: You need to specify a unique document_id (usually pulled from a Logstash event field) so Elasticsearch knows which document to update or create.
• Initialize ctx._source: When performing an upsert (creating a new document), ctx._source will be null—so we explicitly initialize it to an empty object to avoid errors when assigning fields.
• Reference parameters with params.: Inside the script, all dynamic values from Logstash are accessed via the params object, not directly from the Logstash event.

This setup will let you use dynamic values from your Logstash events in the scripted_upsert logic, just like you could with static values.

内容的提问来源于stack exchange，提问作者LosBlancoo