SpringBoot定时任务日志按消息ID拆分至不同文件可行性咨询

Hey there! Both of your proposed approaches—using an Aspect or a custom Logback/SLF4J Appender—are completely feasible, and each has its own sweet spot depending on your specific needs. Let’s dive into the details for each:

This is probably the most straightforward way to achieve your goal, leveraging Spring AOP and SLF4J’s Mapped Diagnostic Context (MDC) to tie logs to your message IDs.

How it works: #

• MDC Context: MDC lets you store thread-local key-value pairs that log frameworks can access when formatting/routing logs. We’ll use it to attach the message ID to the current thread’s context while processing a message.
• Aspect Wrapping: Create an AOP aspect to wrap your queue processing method. Before processing a message, we’ll inject its unique ID into MDC; after processing, we’ll clean up the context to avoid leaks.
• SiftingAppender: Configure Logback’s SiftingAppender to split logs into separate files based on the MDC messageId value.

Step-by-step implementation: #

1. Inject Message ID into MDC via Aspect

   @Aspect
   @Component
   public class MessageIdLoggingAspect {
   
       @Around("execution(* com.yourpackage.QueueProcessingScheduler.processMessage(..))")
       public Object wrapMessageProcessing(ProceedingJoinPoint joinPoint) throws Throwable {
           // Extract the message ID from the method arguments (adjust based on your method signature)
           QueueMessage message = (QueueMessage) joinPoint.getArgs()[0];
           String messageId = message.getUniqueId();
   
           try {
               // Put the ID into MDC
               MDC.put("messageId", messageId);
               // Proceed with the original method execution
               return joinPoint.proceed();
           } finally {
               // Clean up MDC to prevent thread context leaks
               MDC.remove("messageId");
           }
       }
   }
   

2. Configure Logback’s SiftingAppender
   Add this to your logback-spring.xml or logback.xml:

   <appender name="MESSAGE_SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
       <!-- Use the MDC key we set in the aspect -->
       <discriminator>
           <key>messageId</key>
           <!-- Fallback if no ID is present -->
           <defaultValue>unknown-message</defaultValue>
       </discriminator>
       <!-- Define the appender template for each unique message ID -->
       <sift>
           <appender name="FILE-${messageId}" class="ch.qos.logback.core.FileAppender">
               <file>logs/message-${messageId}.log</file>
               <encoder>
                   <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
               </encoder>
           </appender>
       </sift>
   </appender>
   
   <!-- Attach the sifting appender to your logger -->
   <logger name="com.yourpackage" level="INFO" additivity="false">
       <appender-ref ref="MESSAGE_SIFT" />
   </logger>
   

Pros & Notes: #

• Low code intrusion: No changes needed to your core queue processing logic.
• Async caveat: If your scheduler uses async execution (e.g., @Async), you’ll need to manually propagate the MDC context to child threads (Spring’s TaskDecorator can help with this).
• Clean resource management: Logback handles opening/closing file streams automatically.

If you need more control over log splitting logic (like custom file naming rules, automatic log rotation based on message completion, or cleaning up old files), a custom Logback Appender is the way to go.

How it works: #

We’ll create a custom Appender that either pulls the message ID from MDC (similar to the first approach) or parses it directly from log messages. Then, we’ll maintain a map of file appenders, one per unique message ID, and route logs to the correct file.

Step-by-step implementation: #

1. Create the Custom Appender

   @NoArgsConstructor
   public class MessageIdSplitAppender extends UnsynchronizedAppenderBase<ILoggingEvent> {
       // Use a concurrent map to handle thread-safe appender creation
       private final Map<String, FileAppender<ILoggingEvent>> messageAppenders = new ConcurrentHashMap<>();
       private String logDirectory = "logs";
   
       // Optional: Configure log directory via logback config
       public void setLogDirectory(String logDirectory) {
           this.logDirectory = logDirectory;
       }
   
       @Override
       protected void append(ILoggingEvent event) {
           // Get message ID from MDC (or parse from event.getMessage() if needed)
           String messageId = event.getMDCPropertyMap().get("messageId");
           if (messageId == null) {
               messageId = "unknown-message";
           }
   
           // Create or retrieve the appender for this message ID
           FileAppender<ILoggingEvent> appender = messageAppenders.computeIfAbsent(messageId, this::createFileAppender);
           appender.doAppend(event);
       }
   
       private FileAppender<ILoggingEvent> createFileAppender(String messageId) {
           FileAppender<ILoggingEvent> appender = new FileAppender<>();
           appender.setContext(getContext());
           appender.setName("MessageFileAppender-" + messageId);
           appender.setFile(String.format("%s/message-%s.log", logDirectory, messageId));
   
           // Configure encoder
           PatternLayoutEncoder encoder = new PatternLayoutEncoder();
           encoder.setContext(getContext());
           encoder.setPattern("%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n");
           encoder.start();
   
           appender.setEncoder(encoder);
           appender.start();
           return appender;
       }
   
       // Clean up appenders when the app shuts down
       @Override
       public void stop() {
           messageAppenders.values().forEach(Appender::stop);
           super.stop();
       }
   }
   

2. Configure the Custom Appender in Logback

   <appender name="CUSTOM_MESSAGE_SPLIT" class="com.yourpackage.MessageIdSplitAppender">
       <logDirectory>logs/message-logs</logDirectory>
   </appender>
   
   <logger name="com.yourpackage" level="INFO" additivity="false">
       <appender-ref ref="CUSTOM_MESSAGE_SPLIT" />
   </logger>
   

Pros & Notes: #

• Full control: You can add logic like auto-archiving logs when a message is processed, limiting file size per message, or deleting old logs.
• Thread safety: Use ConcurrentHashMap to avoid race conditions when creating appenders.
• Resource cleanup: Don’t forget to stop appenders on shutdown to avoid file handles leaks (as shown in the stop() method).

• Use the Aspect + MDC + SiftingAppender approach if you want a quick, low-maintenance solution with minimal code changes.
• Go with the Custom Appender if you need advanced log management logic that the built-in SiftingAppender can’t handle.

内容的提问来源于stack exchange，提问作者DTodt