AKS集群中Mule 4 Runtime Fabric日志乱序问题求助

问题说明 #

在AKS集群的Mule 4 Runtime Fabric环境中，Logger组件输出的日志出现乱序："Acquisition Req:"文本输出后，被其他日志插入，对应的payload内容延迟显示，当前使用Splunk监控日志，需解决该乱序问题。

当前使用的Log4j配置和Logger组件配置如下：

原Log4j配置 #

<?xml version="1.0" encoding="utf-8"?>
<Configuration>

    <!--These are some of the loggers you can enable. 
        There are several more you can find in the documentation. 
        Besides this log4j configuration, you can also use Java VM environment variables
        to enable other logs like network (-Djavax.net.debug=ssl or all) and 
        Garbage Collector (-XX:+PrintGC). These will be append to the console, so you will 
        see them in the mule_ee.log file. -->

    <Appenders>
        <RollingFile name="file" fileName="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}email-notification.log"
                 filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}notification-%i.log">
            <PatternLayout pattern="%-5p %d [%t] [processor: %X{processorPath}; event: %X{correlationId}] %c: %m%n"/>
            <SizeBasedTriggeringPolicy size="10 MB"/>
            <DefaultRolloverStrategy max="10"/>
        </RollingFile>
    </Appenders>

    <Loggers>
        <!-- Http Logger shows wire traffic on DEBUG -->
        <!--AsyncLogger name="org.mule.service.http.impl.service.HttpMessageLogger" level="DEBUG"/-->
        <AsyncLogger name="org.mule.service.http" level="WARN"/>
        <AsyncLogger name="org.mule.extension.http" level="WARN"/>

        <!-- Mule logger -->
        <AsyncLogger name="org.mule.runtime.core.internal.processor.LoggerMessageProcessor" level="INFO"/>

        <AsyncRoot level="INFO">
            <AppenderRef ref="file"/>
        </AsyncRoot>
    </Loggers>

</Configuration>

原Logger组件配置 #

<logger level="INFO" doc:name="Logger" doc:id="be12c9ac-601e-4283-af6e-0151bb241d0f" message="Acquisition Req: #[payload] "/>

原因分析 #

1. 异步日志导致顺序错乱：当前配置使用了AsyncLogger和AsyncRoot，异步日志会将日志事件放入队列，由独立线程处理输出。不同线程的日志事件可能交错执行，导致同一请求的连续日志（甚至同一条日志的内容）被其他日志插入。
2. payload延迟加载：如果payload是Mule的延迟加载数据流，日志输出时可能会分阶段读取流内容，导致"Acquisition Req:"和payload内容被拆分输出，进一步加剧乱序。

解决方案 #

1. 切换为同步日志（核心解决方法） #

将Log4j配置中的所有AsyncLogger替换为Logger，AsyncRoot替换为Root，确保日志事件同步输出，保证同一线程的日志顺序连续：

修改后的Log4j配置：

<?xml version="1.0" encoding="utf-8"?>
<Configuration>

    <!--These are some of the loggers you can enable. 
        There are several more you can find in the documentation. 
        Besides this log4j configuration, you can also use Java VM environment variables
        to enable other logs like network (-Djavax.net.debug=ssl or all) and 
        Garbage Collector (-XX:+PrintGC). These will be append to the console, so you will 
        see them in the mule_ee.log file. -->

    <Appenders>
        <RollingFile name="file" fileName="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}email-notification.log"
                 filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}notification-%i.log">
            <PatternLayout pattern="%-5p %d [%t] [processor: %X{processorPath}; event: %X{correlationId}] %c: %m%n"/>
            <SizeBasedTriggeringPolicy size="10 MB"/>
            <DefaultRolloverStrategy max="10"/>
        </RollingFile>
    </Appenders>

    <Loggers>
        <!-- Http Logger shows wire traffic on DEBUG -->
        <!--Logger name="org.mule.service.http.impl.service.HttpMessageLogger" level="DEBUG"/-->
        <Logger name="org.mule.service.http" level="WARN"/>
        <Logger name="org.mule.extension.http" level="WARN"/>

        <!-- Mule logger -->
        <Logger name="org.mule.runtime.core.internal.processor.LoggerMessageProcessor" level="INFO"/>

        <Root level="INFO">
            <AppenderRef ref="file"/>
        </Root>
    </Loggers>

</Configuration>

2. 确保日志消息一次性构建完成 #

修改Logger组件的message表达式，强制将payload序列化为字符串后再拼接日志，避免流延迟加载导致的日志拆分：

修改后的Logger配置：

<logger level="INFO" doc:name="Logger" doc:id="be12c9ac-601e-4283-af6e-0151bb241d0f" message="Acquisition Req: #[output application/json --- payload]"/>

提示：如果payload是XML或其他格式，将application/json替换为对应格式即可（如application/xml）。

3. Splunk侧辅助关联（可选） #

如果因性能需求无法切换为同步日志，可在Splunk中通过日志里的correlationId（对应event: %X{correlationId}字段）筛选同一请求的所有日志，即使顺序错乱，也能完整查看单请求的日志链。

内容的提问来源于stack exchange，提问作者kushma gonna