Spring Batch中使用BeanIOFlatFileItemReader跳过CSV表头的方案咨询

问题背景 #

使用BeanIOFlatFileItemReader处理CSV时，发现该Reader没有linesToSkip属性，尝试通过BeanIO配置文件的comments属性忽略表头时，出现Comments require reader.markSupported() to return true itemreader spring batch错误。

相关配置、代码及CSV内容如下：

原BeanIO配置（beanio-configuration.xml）

<stream name="contact-data" format="delimited">
    <parser>
        <property name="delimiter" value="|" />
        <!-- 尝试忽略表头行 -->
        <property name="comments" value="name" />
    </parser>
    <record name="content" class="com.person.Contact">
        <field name="name" />
        <field name="age" />
        <field name="country" />
    </record>       
</stream>

原ItemReader代码

@Bean
@StepScope
public BeanIOFlatFileItemReader<Contact> reader(@Value("#{jobParameters['fileName']}") String fileName) {               
    
    BeanIOFlatFileItemReader<Contact> reader = new BeanIOFlatFileItemReader<>();
    
    try {           
        reader.setUseSpringExceptions(true);
        reader.setResource(new FileSystemResource(fileName));           
        reader.setStreamName(inputContactStreamName);
        reader.setStreamMapping(new ClassPathResource(beanIoConfigurationXmlPath));
        reader.setStreamFactory(StreamFactory.newInstance());
        reader.getLineNumber();         
    } catch (Exception e) {
        log.error("ERROR: An issue occurred in the BeanIO Item Reader:: {} {}", e.getMessage(), e.getStackTrace());
    }
    
    return reader;      
}

CSV文件内容

name|age|country
john|25|USA
Mike|22|CANADA

解决方案 #

方案1：在BeanIO配置中定义表头记录并忽略

直接在BeanIO的stream配置里添加表头记录，设置ignore="true"让BeanIO自动跳过该行：

修改后的beanio-configuration.xml：

<stream name="contact-data" format="delimited">
    <parser>
        <property name="delimiter" value="|" />
    </parser>
    <!-- 定义表头记录，设置ignore=true跳过 -->
    <record name="header" ignore="true">
        <field name="headerName" />
        <field name="headerAge" />
        <field name="headerCountry" />
    </record>
    <!-- 业务数据记录 -->
    <record name="content" class="com.person.Contact">
        <field name="name" />
        <field name="age" />
        <field name="country" />
    </record>       
</stream>

此方式无需修改Java代码，BeanIO会自动识别并跳过第一行表头。

方案2：自定义Reader包装，手动跳过第一行

通过手动读取并跳过第一行表头，再将处理后的流交给BeanIOFlatFileItemReader：

@Bean
@StepScope
public BeanIOFlatFileItemReader<Contact> reader(@Value("#{jobParameters['fileName']}") String fileName) {               
    BeanIOFlatFileItemReader<Contact> reader = new BeanIOFlatFileItemReader<>();
    
    try {           
        reader.setUseSpringExceptions(true);
        // 读取文件资源并跳过表头行
        FileSystemResource resource = new FileSystemResource(fileName);
        BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(resource.getInputStream()));
        bufferedReader.readLine(); // 跳过第一行表头
        
        // 将处理后的流包装为Spring Resource传入Reader
        reader.setResource(new InputStreamResource(bufferedReader));
        
        reader.setStreamName(inputContactStreamName);
        reader.setStreamMapping(new ClassPathResource(beanIoConfigurationXmlPath));
        reader.setStreamFactory(StreamFactory.newInstance());
    } catch (Exception e) {
        log.error("ERROR: An issue occurred in the BeanIO Item Reader:: {} {}", e.getMessage(), e.getStackTrace());
    }
    
    return reader;      
}

注意：需确保流的处理符合Spring Batch生命周期，避免资源泄漏。

方案3：配置底层流支持mark/reset，修复comments配置报错

若坚持使用comments属性，需让BeanIO使用的底层Reader支持markSupported()返回true，可通过配置StreamFactory指定使用BufferedReader：

修改Java代码：

@Bean
@StepScope
public BeanIOFlatFileItemReader<Contact> reader(@Value("#{jobParameters['fileName']}") String fileName) {               
    BeanIOFlatFileItemReader<Contact> reader = new BeanIOFlatFileItemReader<>();
    
    try {           
        reader.setUseSpringExceptions(true);
        reader.setResource(new FileSystemResource(fileName));           
        reader.setStreamName(inputContactStreamName);
        reader.setStreamMapping(new ClassPathResource(beanIoConfigurationXmlPath));
        
        // 创建StreamFactory并配置使用BufferedReader
        StreamFactory streamFactory = StreamFactory.newInstance();
        streamFactory.setReaderFactory(new ReaderFactory() {
            @Override
            public Reader createReader(InputStream in, String charset) throws IOException {
                return new BufferedReader(new InputStreamReader(in, charset));
            }
        });
        reader.setStreamFactory(streamFactory);
    } catch (Exception e) {
        log.error("ERROR: An issue occurred in the BeanIO Item Reader:: {} {}", e.getMessage(), e.getStackTrace());
    }
    
    return reader;      
}

保持原BeanIO配置中的comments属性不变，底层流支持mark/reset后，BeanIO会自动忽略以name开头的表头行，且不会再报错。

内容的提问来源于stack exchange，提问作者Somebody