Log4j2中File Appender运行时变量替换失败问题排查

问题场景 #

配置了Log4j2的控制台输出、滚动日志，同时希望为每个batch任务生成独立日志文件。通过Routing Appender配合ThreadContext设置动态变量，但运行时出现变量未解析的报错：

原Log4j2配置 #

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" packages="com.package1">
    <Properties>
        <Property name="server.log.dir">/opt/tomcat/logs</Property>
    </Properties>
    <ThresholdFilter level="debug"/>
    <Appenders>
        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{account} %c - %msg%n"/>
        </Console>
        <RollingFile name="rolling" fileName="${server.log.dir}/app.log" filePattern="${server.log.dir}/app.log.%d{yyyy-MM-dd-hh-mm}.gz">
            <Policies>
                <SizeBasedTriggeringPolicy size="50 MB"/>
            </Policies>
            <DefaultRolloverStrategy max="10"/>
            <JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json">
                <EventTemplateAdditionalField key="account" format="JSON" value='{"$resolver": "mdc", "key": "account" }'/>
                <EventTemplateAdditionalField key="pipeline" format="JSON" value='{"$resolver": "mdc", "key": "pipeline" }'/>
            </JsonTemplateLayout>
        </RollingFile>
        <Routing name="batchLogging">
            <!-- here is the dynamic log file for each activity -->
            <Routes pattern="$${ctx:activity}">
                <Route key="batch">
                    <File name="batchLogger"
                                 filename="${server.log.dir}/$${ctx:batch_log}"
                                 append="false">
                        <PatternLayout pattern="%d %-5p %X{account} %X{pipeline} [%t] %c - %m%n"/>
                        <DynamicThresholdFilter key="batch_level" defaultTreshold="debug" onMatch="ACCEPT">
                            <KeyValuePair key="fatal" value="FATAL"/>
                            <KeyValuePair key="error" value="ERROR"/>
                            <KeyValuePair key="warn" value="WARN"/>
                            <KeyValuePair key="info" value="INFO"/>
                            <KeyValuePair key="debug" value="DEBUG"/>
                        </DynamicThresholdFilter>
                    </File>
                </Route>
            </Routes>
        </Routing>
    </Appenders>
    <Loggers>
        <Logger name="org.springframework" level="warn"/>
        <Logger name="org.springframework.jdbc.core.simple" level="debug"/>
        <Logger name="org.springframework.web.filter.CommonsRequestLoggingFilter" level="debug"/>
        <Logger name="com.amazonaws" level="warn"/>
        <Logger name="org.apache.zookeeper" level="warn"/>
        <Logger name="org.apache.hadoop" level="warn"/>
        <Logger name="org.apache.solr" level="warn" additivity="false"/>
        <Logger name="org.apache.http" level="warn"/>
        <Logger name="ro.isdc" level="info"/>
        <Logger name="org.xhtmlrenderer" level="info"/>

        <Root level="debug">
            <AppenderRef ref="STDOUT"/>
            <AppenderRef ref="rolling"/>
            <AppenderRef ref="batchLogging"/>
        </Root>
    </Loggers>
</Configuration>

原Java代码 #

public BatchResult call() throws Exception {
    String batchFilename = batch.getAccountId().val() + "/batches/" + batch.getId() + "/" + batch.getLogFilename();
    try {
         ThreadContext.put("activity", "batch");
         ThreadContext.put("batch_log", batchFilename );
         ThreadContext.put("batch_level", options.containsKey("logLevel") ? (String)options.get("logLevel") : Level.DEBUG.name() );
         return importData(currentUser, filename, scriptName, batch, sendEmail, triggerBuilders, options);
    } finally {
         ThreadContext.remove("batch_log");
         ThreadContext.remove("batch_level");
         ThreadContext.remove("activity");
         File logDir = new File(etlEnvironment.getServerDataHome(), "archive");
         uploadLogs(new File( logDir, batchFilename), batch);
    }
}

报错信息 #

ERROR StatusConsoleListener Could not create plugin of type class org.apache.logging.log4j.core.appender.FileAppender for element File: java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.FileManager$FileManagerFactory@551f811a] unable to create manager for [/opt/tomcat/logs/archive/${ctx:batch_log}] with data [org.apache.logging.log4j.core.appender.FileManager$FactoryData@203052b9]
 java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.FileManager$FileManagerFactory@551f811a] unable to create manager for [/opt/tomcat/logs/archive/${ctx:batch_log}] with data [org.apache.logging.log4j.core.appender.FileManager$FactoryData@203052b9]

错误原因 #

1. 双重$符号误用：$${ctx:batch_log}会让Log4j2将其解析为字符串${ctx:batch_log}，而非读取ThreadContext中的变量值。
2. Routing匹配规则不合理：基于固定的activity值匹配，导致所有batch任务共用同一个File Appender，无法区分不同的日志文件路径。
3. 目录未自动创建：batchFilename包含子目录结构，而默认File Appender不会自动创建目录，导致文件创建失败。

解决方案 #

修改后的Log4j2配置 #

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" packages="com.package1" monitorInterval="30">
    <Properties>
        <Property name="server.log.dir">/opt/tomcat/logs</Property>
    </Properties>
    <ThresholdFilter level="debug"/>
    <Appenders>
        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %X{account} %c - %msg%n"/>
        </Console>
        <RollingFile name="rolling" fileName="${server.log.dir}/app.log" filePattern="${server.log.dir}/app.log.%d{yyyy-MM-dd-hh-mm}.gz">
            <Policies>
                <SizeBasedTriggeringPolicy size="50 MB"/>
            </Policies>
            <DefaultRolloverStrategy max="10"/>
            <JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json">
                <EventTemplateAdditionalField key="account" format="JSON" value='{"$resolver": "mdc", "key": "account" }'/>
                <EventTemplateAdditionalField key="pipeline" format="JSON" value='{"$resolver": "mdc", "key": "pipeline" }'/>
            </JsonTemplateLayout>
        </RollingFile>
        <Routing name="batchLogging">
            <!-- 基于唯一的batch日志路径匹配，每个batch生成独立Appender -->
            <Routes pattern="$${ctx:batch_log}">
                <Route>
                    <File name="batchLogger-${ctx:batch_log}"
                          filename="${server.log.dir}/${ctx:batch_log}"
                          append="false"
                          createOnDemand="true"> <!-- 自动创建所需子目录 -->
                        <PatternLayout pattern="%d %-5p %X{account} %X{pipeline} [%t] %c - %m%n"/>
                        <DynamicThresholdFilter key="batch_level" defaultThreshold="debug" onMatch="ACCEPT">
                            <KeyValuePair key="fatal" value="FATAL"/>
                            <KeyValuePair key="error" value="ERROR"/>
                            <KeyValuePair key="warn" value="WARN"/>
                            <KeyValuePair key="info" value="INFO"/>
                            <KeyValuePair key="debug" value="DEBUG"/>
                        </DynamicThresholdFilter>
                    </File>
                </Route>
            </Routes>
        </Routing>
    </Appenders>
    <Loggers>
        <Logger name="org.springframework" level="warn"/>
        <Logger name="org.springframework.jdbc.core.simple" level="debug"/>
        <Logger name="org.springframework.web.filter.CommonsRequestLoggingFilter" level="debug"/>
        <Logger name="com.amazonaws" level="warn"/>
        <Logger name="org.apache.zookeeper" level="warn"/>
        <Logger name="org.apache.hadoop" level="warn"/>
        <Logger name="org.apache.solr" level="warn" additivity="false"/>
        <Logger name="org.apache.http" level="warn"/>
        <Logger name="ro.isdc" level="info"/>
        <Logger name="org.xhtmlrenderer" level="info"/>

        <Root level="debug">
            <AppenderRef ref="STDOUT"/>
            <AppenderRef ref="rolling"/>
            <AppenderRef ref="batchLogging"/>
        </Root>
    </Loggers>
</Configuration>

关键修改说明 #

• Routing匹配规则调整：将pattern改为$${ctx:batch_log}，每个不同的日志路径对应独立的Route实例，确保每个batch任务拥有专属日志文件。
• 变量解析修正：filename改为${server.log.dir}/${ctx:batch_log}，直接读取ThreadContext变量值，避免双重$导致的字符串保留问题。
• 自动创建目录：添加createOnDemand="true"，让File Appender自动生成日志所需的子目录。
• 唯一Appender名称：将Appender命名为batchLogger-${ctx:batch_log}，避免动态创建的Appender出现名称冲突。

Java代码优化（可选） #

提前创建目录（若不依赖createOnDemand），并确保ThreadContext变量清理顺序正确：

public BatchResult call() throws Exception {
    String batchFilename = batch.getAccountId().val() + "/batches/" + batch.getId() + "/" + batch.getLogFilename();
    // 提前创建日志目录（可选，若使用createOnDemand可省略）
    File logFile = new File("/opt/tomcat/logs", batchFilename);
    if (!logFile.getParentFile().exists()) {
        logFile.getParentFile().mkdirs();
    }
    try {
         ThreadContext.put("batch_log", batchFilename );
         ThreadContext.put("batch_level", options.containsKey("logLevel") ? (String)options.get("logLevel") : Level.DEBUG.name() );
         ThreadContext.put("activity", "batch");
         return importData(currentUser, filename, scriptName, batch, sendEmail, triggerBuilders, options);
    } finally {
         // 按添加顺序逆序清理，避免变量残留
         ThreadContext.remove("activity");
         ThreadContext.remove("batch_level");
         ThreadContext.remove("batch_log");
         File logDir = new File(etlEnvironment.getServerDataHome(), "archive");
         uploadLogs(new File( logDir, batchFilename), batch);
    }
}

额外注意事项 #

• Log4j2版本要求：createOnDemand属性需要Log4j2 2.10.0及以上版本支持。
• 线程池场景：若使用线程池执行任务，需在任务开始前清理ThreadContext变量，避免线程复用导致的变量污染。
• 日志清理策略：动态生成的日志文件不会自动清理，可配置定时任务或替换为RollingFile Appender实现滚动和过期清理。

内容的提问来源于stack exchange，提问作者chubbsondubs