编写提取键值对的Grok表达式求助：指标输出解析需求

Absolutely! Grok is made for exactly this kind of structured parsing from unstructured metrics output. The exact pattern you’ll use depends on how your metrics are formatted, but I’ll walk you through common scenarios and actionable examples to get you started.

Case 1: Space-separated key=value pairs #

Suppose your metrics look like this:

cpu_usage=23.4 memory_used=1024MB disk_free=50% uptime=1200s

If you know all the keys in advance, you can write a direct Grok pattern to capture each pair with proper data types:

%{DATA:cpu_key}=%{NUMBER:cpu_usage:float} %{DATA:memory_key}=%{DATA:memory_used} %{DATA:disk_key}=%{DATA:disk_free} %{DATA:uptime_key}=%{NUMBER:uptime:int}s

If your keys are dynamic (you don’t know all of them upfront), define a custom pattern for a single key-value pair first:

KV_PAIR %{DATA:key}=%{DATA:value}

Then use this repeating pattern to match all pairs in the line:

%{KV_PAIR} (?:%{KV_PAIR})*

Note: Grok will overwrite fields if the same key appears multiple times. For better handling of dynamic pairs, consider combining Grok with the kv filter (common in tools like Logstash) after capturing the full set of pairs into a single field.

Case 2: Comma-separated key: value pairs #

For metrics formatted like:

cpu_usage: 23.4, memory_used: 1024MB, disk_free: 50%

A targeted Grok pattern would be:

%{DATA:cpu_key}: %{NUMBER:cpu_usage:float}, %{DATA:memory_key}: %{DATA:memory_used}, %{DATA:disk_key}: %{DATA:disk_free}

For dynamic keys here, create a custom colon-separated pair pattern:

KV_COLON_PAIR %{DATA:key}: %{DATA:value}

Then match all pairs with:

%{KV_COLON_PAIR}(?:, %{KV_COLON_PAIR})*

Case 3: Bracketed key-value pairs #

If your metrics use brackets and separators like pipes:

[cpu_usage] 23.4 | [memory_used] 1024MB | [disk_free] 50%

Use this Grok pattern for known keys:

\[%{DATA:cpu_key}\] %{NUMBER:cpu_usage:float} \| \[%{DATA:memory_key}\] %{DATA:memory_used} \| \[%{DATA:disk_key}\] %{DATA:disk_free}

For dynamic bracketed pairs, define:

BRACKET_KV_PAIR \[%{DATA:key}\] %{DATA:value}

Then match all with:

%{BRACKET_KV_PAIR}(?: \| %{BRACKET_KV_PAIR})*

• Test your patterns with a Grok debugger (most tools like Logstash or Elastic Stack have built-in ones) to tweak and validate without trial-and-error in production.
• Always specify data types (like :float or :int) after field names—this ensures your metrics are stored as numbers, not strings, which is critical for analysis.
• For messy or variable formats, first use Grok to isolate the metrics section into a single field, then use the kv filter to split it into key-value pairs. For example, in Logstash:

  filter {
    grok {
      match => { "message" => "System Metrics: %{GREEDYDATA:metric_data}" }
    }
    kv {
      source => "metric_data"
      field_split => " "
      value_split => "="
    }
  }
  

  This takes input like System Metrics: cpu_usage=23.4 memory_used=1024MB and outputs structured fields cpu_usage: 23.4 and memory_used: 1024MB.

内容的提问来源于stack exchange，提问作者Dushyanth Nagaraju