Let's break down your two questions with practical, actionable steps using your sample log document as a reference.

1. Extracting Fields & Moving Nested Data to a New Parent Field in Logstash #

Logstash uses bracket notation ([parent][child]) to handle nested field structures, making it straightforward to group extracted fields under a new parent. Here's how to do it:

Suppose you've already pulled out target fields (via grok, dissect, or another filter) and want to organize them under a new parent like event_metadata. Here's a sample filter config:

filter {
  # First, extract your fields (example using grok if values live in the message)
  grok {
    match => { "message" => "some_pattern_%{DATA:user_id}_%{NUMBER:session_duration}" }
  }

  # Create the new parent field and move extracted fields into it
  mutate {
    # Use [parent][child] syntax to build the nested structure
    add_field => { "[event_metadata][user_id]" => "%{user_id}" }
    add_field => { "[event_metadata][session_duration]" => "%{session_duration}" }

    # Optional: Clean up by removing the original top-level fields
    remove_field => ["user_id", "session_duration"]
  }
}

This will result in a clean nested structure in your Elasticsearch document:

"_source": {
  // ... other existing fields ...
  "event_metadata": {
    "user_id": "user_123",
    "session_duration": 456
  }
}

2. Capturing LON/LAT & Making Them Geospatial Data in Elasticsearch #

To turn your log's LON and LAT values into a usable geospatial field, you need two key steps: extract and format the coordinates in Logstash, then map the field as a geo_point in Elasticsearch.

Step 1: Logstash Filter to Process Coordinates #

Your sample message is "LON: 12.5, LAT: 42" — use this filter to pull out the numbers, convert them to floats, and package them into a structure Elasticsearch recognizes as geospatial data:

filter {
  # Extract LON and LAT values from the message
  grok {
    match => { "message" => "LON: %{NUMBER:lon}, LAT: %{NUMBER:lat}" }
  }

  # Convert extracted string values to floats (required for geo_point type)
  mutate {
    convert => {
      "lon" => "float"
      "lat" => "float"
    }
  }

  # Package into a nested `location` field (Elasticsearch's preferred format)
  mutate {
    add_field => { "[location][lat]" => "%{lat}" }
    add_field => { "[location][lon]" => "%{lon}" }

    # Optional: Remove the original top-level lat/lon fields to clean up
    remove_field => ["lat", "lon"]
  }
}

Step 2: Elasticsearch Index Template for Geospatial Mapping #

Elasticsearch won't automatically treat location as a geospatial field — you need to define a mapping via an index template. This ensures every new memo-logstash-* index uses the correct type:

PUT _index_template/memo-logstash-geo-template
{
  "index_patterns": ["memo-logstash-*"],
  "mappings": {
    "properties": {
      "location": {
        "type": "geo_point"
      }
    }
  }
}

After applying this template and processing your logs, your Elasticsearch document will include a usable geospatial field:

{
  "_index": "memo-logstash-2018.05",
  "_source": {
    "type": "elktest",
    "message": "LON: 12.5, LAT: 42",
    "@timestamp": "2018-05-09T10:44:09.046Z",
    "host": "f6f9fd66cd6c",
    "path": "/usr/share/logstash/logs/docker-elk-master.log",
    "@version": "1",
    "location": {
      "lat": 42.0,
      "lon": 12.5
    }
  }
}

You can now use this location field for Kibana map visualizations, geospatial queries (like finding logs within a specific radius), and other location-based analytics.

内容的提问来源于stack exchange，提问作者Alessio Frabotta