本地localhost:9200环境下Logstash无法输出至Elasticsearch求助

Let's work through this issue step by step. First, I spot two key problems with your current setup:

1. Your test.conf doesn't include an Elasticsearch output block — that's why your data isn't being sent to your local ES instance at localhost:9200.
2. The existing date filter won't function correctly because you haven't first extracted the timestamp field from your raw Nginx log line.

Revised Logstash Config (test.conf) #

Here's an updated configuration that addresses both issues, plus proper parsing for your Nginx access log format:

input {
  file {
    path => "C:/logs/nginxAccess.log"
    start_position => "beginning"
    # Prevent Logstash from remembering previous read positions (useful for testing)
    sincedb_path => "NUL"
  }
}

filter {
  # Parse raw Nginx log into structured fields using Grok
  grok {
    match => { "message" => '%{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{URIPATHPARAM:request} %{DATA:http_version}" %{NUMBER:status} %{NUMBER:bytes_sent} "%{DATA:referrer}" "%{DATA:user_agent}"' }
  }

  # Convert extracted timestamp to Elasticsearch's native @timestamp field
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    target => "@timestamp"
  }
}

output {
  # Send data to your local Elasticsearch instance
  elasticsearch {
    hosts => ["localhost:9200"]
    # Define a custom index name (adjust as needed)
    index => "nginx-access-%{+YYYY.MM.dd}"
  }

  # Keep stdout output for debugging
  stdout { codec => rubydebug }
}

Key Explanations: #

• Input: Added sincedb_path => "NUL" to ensure Logstash re-reads the entire log file on each run (great for testing; remove this in production if you want to track read positions).
• Filter:
  • The grok pattern matches your Nginx log format and extracts fields like clientip, request, status, and crucially, timestamp.
  • The date filter converts the human-readable timestamp into Elasticsearch's standardized @timestamp field, which is required for proper time-based indexing.
• Output: The new elasticsearch block sends parsed data to your local localhost:9200 instance, with a daily rotating index name for easier management.

Testing & Verification: #

1. Run Logstash with the updated config:

   logstash -f test.conf
   

2. Check if the index is created in Elasticsearch by running this command in your terminal:

   curl localhost:9200/_cat/indices?v
   

   You should see an index named something like nginx-access-2024.05.20 (matching the current date).
3. You can also query the index to confirm data is present:

   curl localhost:9200/nginx-access-*/_search?q=*&pretty
   

内容的提问来源于stack exchange，提问作者Shiva