如何在Fluentd中配置Filter实现包含INFO关键字的日志记录删除

Hey there, to drop entire log records where the log field includes the "INFO" keyword, you can use Fluentd's built-in grep filter plugin—it's perfect for this kind of record filtering. Here's a straightforward configuration that does exactly what you need:

<filter **>
  @type grep
  <exclude>
    key log
    pattern /INFO/
  </exclude>
</filter>

Let me break down what each part does: #

• <filter **>: This applies the filter to all incoming log records (the ** wildcard matches any tag). If you only want to target specific logs (like the Kubernetes/CoreDNS logs in your example), adjust the tag pattern to something like other_service.kubernetes.** instead.
• @type grep: This tells Fluentd we're using the grep plugin, which lets you include or exclude records based on field content patterns.
• <exclude> block: This is the key part—it instructs Fluentd to discard any record that matches the rules inside:
  • key log: Targets the log field in your log records (the field that contains the actual log message text).
  • pattern /INFO/: A regular expression that matches any occurrence of "INFO" in the log field. This is case-sensitive by default; if you need to match "info" or "Info" too, use /INFO/i (the i flag enables case-insensitive matching).

For more precise matching (like your sample log's [INFO] marker): #

If you want to avoid accidentally matching "INFO" in other contexts (like a URL or username), you can target the exact [INFO] string from your sample log. Since square brackets are special characters in regular expressions, you'll need to escape them:

<filter **>
  @type grep
  <exclude>
    key log
    pattern /\[INFO\]/
  </exclude>
</filter>

Once you add this filter to your Fluentd config, any record where the log field matches the pattern will be dropped entirely before it reaches your output destination (like Elasticsearch in your example).

内容的提问来源于stack exchange，提问作者koushick patel