Spring Batch文件分区导入报错:FlatFileItemReader无parameterValues属性
Hey there! Let's tackle your problem head-on. First, let's break down why you're seeing that error, then look at the right tools for file partitioning in Spring Batch.
Why the Error Occurs
The org.springframework.beans.NotWritablePropertyException is clear about the issues:
parameterValuesisn't a valid property ofFlatFileItemReader: This property belongs to JDBC-based readers likeJdbcCursorItemReader, not file readers. You likely mixed up configurations from different reader types.rowMapperis also invalid here:FlatFileItemReaderuses alineMapperto parse CSV lines, whilerowMapperis for mapping database rows to POJOs—another mismatch in your setup.
Recommended Readers for File Partitioning
For partitioning CSV file imports, you have two solid approaches depending on your file size and use case:
1. FlatFileItemReader with RangePartitioner (For Single Large File)
If you're working with one large CSV file, use RangePartitioner to split the file into row ranges. Then configure FlatFileItemReader to read only its assigned range using two built-in properties:
currentItemCount: Skips the first N rows (matches yourstartingIndex)maxItemCount: Limits the number of rows to read (calculated asendingIndex - startingIndex + 1)
Corrected Reader Configuration
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step"> <property name="resource" value="classpath:ExamResult.txt" /> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="fieldSetMapper"> <bean class="com.websystique.springbatch.ExamResultFieldSetMapper" /> </property> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="delimiter" value="|" /> </bean> </property> </bean> </property> <!-- Use StepExecutionContext values to control row range --> <property name="currentItemCount" value="#{stepExecutionContext['startingIndex']}" /> <property name="maxItemCount" value="#{stepExecutionContext['endingIndex'] - stepExecutionContext['startingIndex'] + 1}" /> </bean>
Example RangePartitioner Setup
<bean id="rangePartitioner" class="org.springframework.batch.core.partition.support.RangePartitioner"> <!-- Number of partitions (adjust based on your needs) --> <property name="gridSize" value="2" /> </bean>
2. MultiResourceItemReader (For Pre-Split Files)
If your file is extremely large (10GB+), it's more efficient to split it into smaller, equal-sized files first (using a pre-processing tasklet or external tool). Then use MultiResourceItemReader to assign each small file to a partition.
Sample Configuration
<bean id="multiResourceItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader" scope="step"> <!-- Get the assigned file resource from StepExecutionContext --> <property name="resources" value="#{stepExecutionContext['fileResource']}" /> <property name="delegate"> <bean class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="lineMapper"> <!-- Same lineMapper as before --> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="fieldSetMapper"> <bean class="com.websystique.springbatch.ExamResultFieldSetMapper" /> </property> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"> <property name="delimiter" value="|" /> </bean> </property> </bean> </property> </bean> </property> </bean>
Your partitioner would populate fileResource in each Step's execution context with the path to one of the split files.
Key Takeaways
- Remove
parameterValuesandrowMapperfrom yourFlatFileItemReaderconfig—they don't belong here. - Use
currentItemCount+maxItemCountwithRangePartitionerfor single large files. - Use
MultiResourceItemReaderfor pre-split smaller files (better performance for massive datasets).
内容的提问来源于stack exchange,提问作者user2991123

