FluentD 1.16.3+的record_transformer过滤器支持多<record>段吗？

核心结论 #

FluentD 1.16.3及以上版本的record_transformer插件不支持多个<record>配置段——每个filter块内只会生效最后一个<record>的配置，前面的所有<record>设置都会被覆盖，这就是你观察到只有最后一段生效的原因。

问题分析 #

你尝试的两种配置都存在局限性：

1. 多<record>段：后段会完全覆盖前段的字段设置，无法实现多条件分别匹配的需求；
2. 单<record>+elsif：一旦匹配到第一个符合条件的分支，后续判断就会终止，无法处理单条日志可能匹配多个条件的场景（或确保所有条件都被检查）。

解决方案 #

方案1：拆分多个独立的record_transformer filter

将每个条件拆分为单独的filter块，通过<condition>过滤出对应日志后再应用字段转换，避免互相覆盖：

<filter authentication>
  @type record_transformer
  enable_ruby
  remove_keys log
  <record>
    message ${record["log"]}
    audit.category Login
    audit.event ${record["log"].include?("Invalid credentials") ? "Failed Login" : "Successful Login"}
    audit.dashboards.authentication true
  </record>
  <condition>
    <ruby>
      # 仅处理未匹配其他分类的日志
      !record["log"].include?("Anonymous access not allowed") && 
      !record["log"].include?("SRCH base=") && 
      !record["log"].include?("RESULT err=")
    </ruby>
  </condition>
</filter>

<filter authentication>
  @type record_transformer
  enable_ruby
  <record>
    message ${record["log"]}
    audit.category Anonymous_Login
    audit.event "LDAP Anonymous Access Attempted"
    audit.dashboards.authentication true
  </record>
  <condition>
    <ruby>
      record["log"].include?("Anonymous access not allowed")
    </ruby>
  </condition>
</filter>

<filter authentication>
  @type record_transformer
  enable_ruby
  <record>
    message ${record["log"]}
    audit.category LDAP_Search
    audit.event "LDAP Search Action"
    audit.dashboards.authentication true
  </record>
  <condition>
    <ruby>
      record["log"].include?("SRCH base=")
    </ruby>
  </condition>
</filter>

<filter authentication>
  @type record_transformer
  enable_ruby
  <record>
    message ${record["log"]}
    audit.category LDAP_Search_Result
    audit.event "LDAP Search Result"
    audit.dashboards.authentication true
  </record>
  <condition>
    <ruby>
      record["log"].include?("RESULT err=")
    </ruby>
  </condition>
</filter>

方案2：单filter内用Ruby逻辑实现多条件判断

在单个<record>段中嵌入完整的Ruby逻辑，处理所有条件分支，包括默认情况：

<filter authentication>
  @type record_transformer
  enable_ruby
  remove_keys log
  <record>
    message ${record["log"]}
    audit.category ${
      case record["log"]
      when /Invalid credentials/
        "Login"
      when /Anonymous access not allowed/
        "Anonymous_Login"
      when /SRCH base=/
        "LDAP_Search"
      when /RESULT err=/
        "LDAP_Search_Result"
      else
        "Login"
      end
    }
    audit.event ${
      case record["log"]
      when /Invalid credentials/
        "Failed Login"
      when /Anonymous access not allowed/
        "LDAP Anonymous Access Attempted"
      when /SRCH base=/
        "LDAP Search Action"
      when /RESULT err=/
        "LDAP Search Result"
      else
        "Successful Login"
      end
    }
    audit.dashboards.authentication true
  </record>
</filter>

这种方式更简洁，适合日志只会匹配单个条件的场景，同时通过case语句清晰处理所有分支。

内容的提问来源于stack exchange，提问作者AKS