如何使用Spring Batch从多行数据构建复杂POJO

Hey there! As someone who’s helped many Spring Batch newbies work through exactly these kinds of workflows, let’s break this down step by step. You’ve got a clear pipeline in mind—let’s turn that into working code and actionable best practices.

First, make sure your build file includes all the necessary Spring Batch, JPA, and web dependencies. For Maven, add these to your pom.xml:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <!-- Add your database driver (e.g., PostgreSQL, H2) -->
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>runtime</scope>
    </dependency>
</dependencies>

This is often the trickiest part for newbies—combining multiple lines from your text file into one POJO. Let’s assume your file has groups of 3 lines that map to an Order (adjust the line count to match your actual format):

First, Define Your POJO (and JPA Entity) #

@Entity
@Table(name = "orders")
public class Order {
    @Id
    private String orderId;
    private LocalDate orderDate;
    private String customerName;
    private String productSku;

    // Getters, setters, and no-arg constructor required for JPA
}

Next, Build a Custom ItemReader to Aggregate Rows #

Instead of reading one line at a time, we’ll read groups of lines and assemble them into an Order:

@Component
public class AggregatedOrderReader implements ItemReader<Order> {
    private final FlatFileItemReader<String> lineReader;
    private int lineCounter = 0;

    public AggregatedOrderReader() {
        this.lineReader = new FlatFileItemReaderBuilder<String>()
                .resource(new ClassPathResource("orders.txt"))
                .lineMapper(new PassThroughLineMapper()) // Read raw lines first
                .build();
    }

    @Override
    public Order read() throws Exception {
        // Read 3 lines per Order (adjust this number to match your file structure)
        if (lineCounter % 3 == 0) {
            String headerLine = lineReader.read();
            if (headerLine == null) return null; // End of file

            String customerLine = lineReader.read();
            String productLine = lineReader.read();

            // Parse fixed-length fields from each line
            Order order = new Order();
            order.setOrderId(headerLine.substring(0, 6).trim()); // Positions 1-6 = order ID
            order.setOrderDate(LocalDate.parse(headerLine.substring(6, 16).trim())); // Positions 7-16 = date
            order.setCustomerName(customerLine.substring(0, 20).trim()); // Positions 1-20 = customer name
            order.setProductSku(productLine.substring(0, 10).trim()); // Positions 1-10 = product SKU

            lineCounter += 3;
            return order;
        }
        return null;
    }
}

Pro Tip: For cleaner parsing, use Spring’s FixedLengthTokenizer to map line segments to DTOs, then aggregate those DTOs into your POJO in an ItemProcessor instead. This keeps your reader focused on reading lines, not business logic.

Use Spring’s WebClient (modern alternative to RestTemplate) to send your Order to the external application. We’ll do this in an ItemProcessor:

@Component
public class OrderRestProcessor implements ItemProcessor<Order, Order> {
    private final WebClient webClient;

    public OrderRestProcessor(WebClient.Builder webClientBuilder) {
        this.webClient = webClientBuilder.baseUrl("http://external-app/api/orders").build();
    }

    @Override
    public Order process(Order order) throws Exception {
        // Send POST request and handle errors
        ResponseEntity<Void> response = webClient.post()
                .bodyValue(order)
                .retrieve()
                .toBodilessEntity()
                .block();

        if (response.getStatusCode().isError()) {
            throw new RuntimeException("Failed to send order to external app: " + response.getStatusCode());
        }

        return order; // Pass the order to the next step (persistence)
    }
}

Error Handling Note: Add retry logic for transient REST failures using Spring Retry—just add the spring-boot-starter-aop and spring-retry dependencies, then annotate your process method with @Retryable.

First, create a repository interface for your Order entity:

public interface OrderRepository extends JpaRepository<Order, String> {
}

Then build an ItemWriter to save orders to the database:

@Component
public class OrderJpaWriter implements ItemWriter<Order> {
    private final OrderRepository orderRepository;

    public OrderJpaWriter(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    @Override
    public void write(List<? extends Order> items) throws Exception {
        orderRepository.saveAll(items); // Batch-save for efficiency
    }
}

Finally, configure your job and step to connect all the components:

@Configuration
@EnableBatchProcessing
public class OrderBatchConfig {
    private final JobBuilderFactory jobBuilderFactory;
    private final StepBuilderFactory stepBuilderFactory;
    private final AggregatedOrderReader orderReader;
    private final OrderRestProcessor orderProcessor;
    private final OrderJpaWriter orderJpaWriter;
    // Add your post-persistence writer here (e.g., another file, notification service)

    public OrderBatchConfig(JobBuilderFactory jobBuilderFactory, StepBuilderFactory stepBuilderFactory,
                            AggregatedOrderReader orderReader, OrderRestProcessor orderProcessor,
                            OrderJpaWriter orderJpaWriter) {
        this.jobBuilderFactory = jobBuilderFactory;
        this.stepBuilderFactory = stepBuilderFactory;
        this.orderReader = orderReader;
        this.orderProcessor = orderProcessor;
        this.orderJpaWriter = orderJpaWriter;
    }

    @Bean
    public OrderProcessingStep() {
        return stepBuilderFactory.get("orderProcessingStep")
                .<Order, Order>chunk(10) // Process 10 orders at a time (adjust chunk size)
                .reader(orderReader)
                .processor(orderProcessor)
                .writer(orderJpaWriter)
                // Add your post-persistence writer here using CompositeItemWriter if needed:
                // .writer(new CompositeItemWriterBuilder<Order>().delegates(orderJpaWriter, postPersistenceWriter).build())
                .faultTolerant() // Enable error handling/retry
                .retry(RuntimeException.class)
                .retryLimit(3)
                .build();
    }

    @Bean
    public Job orderProcessingJob() {
        return jobBuilderFactory.get("orderProcessingJob")
                .incrementer(new RunIdIncrementer()) // Ensure job runs with unique IDs
                .start(orderProcessingStep())
                .build();
    }
}

• Testing: Use @SpringBatchTest to write unit tests for your reader, processor, and writer individually before testing the full job.
• Transaction Management: Spring Batch handles transactions at the chunk level by default—if you need to ensure REST calls and persistence are atomic, adjust the transaction boundaries in your step configuration.
• Logging: Add detailed logging to each component to debug issues with file parsing, REST calls, or persistence.

内容的提问来源于stack exchange，提问作者Enrico Bergamo