Spring Integration：Splitter与Aggregator实现文件拆分聚合及分类归档

Looks like you want to extend your existing file processing workflow to persist both valid and corrupted lines separately—great idea for auditability and debugging! Here's a practical implementation plan with code examples to get you there:

First, make sure the target directories for completed and corrupted lines exist. Trying to write files to non-existent paths will throw errors, so add logic to create them upfront:

import java.nio.file.*;

// Define your output directories
Path completedDir = Paths.get("./processed/completed");
Path corruptDir = Paths.get("./processed/corrupted");

// Create directories if they don't exist
try {
    Files.createDirectories(completedDir);
    Files.createDirectories(corruptDir);
} catch (Exception e) {
    System.err.println("Failed to create output directories: " + e.getMessage());
    throw new RuntimeException(e);
}

Modify your existing line-reading loop to:

• Check if a line is corrupted (using your existing filter logic)
• Write corrupted lines to the dedicated corrupt file immediately
• For valid lines, call your WebService, then write to the completed file (adjust this if you want to write before the WebService call—depends on your definition of "completed")

Here's how that looks in practice:

import java.io.BufferedWriter;
import java.io.IOException;

// Assume you have your FileSplitter and WebService call methods ready
String inputFileName = "data.txt";

// Initialize writers for output files (use try-with-resources to auto-close)
try (BufferedWriter completedWriter = Files.newBufferedWriter(
         completedDir.resolve(inputFileName + "_completed.txt"),
         StandardOpenOption.CREATE, StandardOpenOption.APPEND
     );
     BufferedWriter corruptWriter = Files.newBufferedWriter(
         corruptDir.resolve(inputFileName + "_corrupted.txt"),
         StandardOpenOption.CREATE, StandardOpenOption.APPEND
     );
     FileSplitter splitter = new FileSplitter(Paths.get("./input/" + inputFileName))) {

    String line;
    while ((line = splitter.readLine()) != null) {
        // 1. Check if line is corrupted (your existing filter)
        if (isCorrupted(line)) {
            corruptWriter.write(line);
            corruptWriter.newLine();
            continue;
        }

        // 2. Process valid line with WebService
        try {
            callYourWebService(line);
            
            // 3. Write to completed file only if WebService call succeeds
            completedWriter.write(line);
            completedWriter.newLine();
        } catch (WebServiceException e) {
            // Optional: Handle WebService failures (e.g., write to a "failed" directory)
            System.err.println("WebService failed for line: " + line + " | Error: " + e.getMessage());
            // failedWriter.write(line);
            // failedWriter.newLine();
        }
    }
} catch (IOException e) {
    System.err.println("File processing failed: " + e.getMessage());
}

// Your existing line validation method
private boolean isCorrupted(String line) {
    // Example checks: empty line, missing required fields, invalid format
    return line == null || line.trim().isEmpty() || !line.matches("^\\d+,\\w+,\\d{4}$");
}

// Your existing WebService call method
private void callYourWebService(String line) throws WebServiceException {
    // WebService invocation logic here
}

• Resource Management: Always use try-with-resources (like in the example) to ensure writers and file splitters are closed properly, even if an error occurs.
• File Naming: Adjust the output file names to fit your needs—you might want to include timestamps (e.g., data_20240520_completed.txt) to avoid overwriting files.
• Batch Writing: If you're processing large files, consider batching lines (e.g., write 100 lines at a time) to reduce IO overhead.
• Concurrency: If you're processing lines in parallel, add synchronization around the writers to avoid race conditions.

内容的提问来源于stack exchange，提问作者Ahmed Ali