Spring Batch读取含逗号CSV文件列识别错误的配置咨询

你的问题核心是DelimitedLineTokenizer默认会把所有逗号识别为列分隔符，哪怕逗号在字段内容里。要解决这个问题，确实需要通过setQuoteCharacter()配置引号字符，让阅读器把被引号包裹的内容当作完整字段，忽略内部的逗号。

关键修改步骤 #

1. 调整输入CSV格式：带逗号的字段必须用引号包裹，比如你的输入要改成：

   column1,column2
   steve,"john,bill"
   

2. 在代码中配置引号识别规则：在创建DelimitedLineTokenizer后，添加引号字符的配置。

修改后的核心代码片段 #

DelimitedLineTokenizer delimitedLineTokenizer = new DelimitedLineTokenizer(",");
// 添加这一行，指定双引号为字段包裹符
delimitedLineTokenizer.setQuoteCharacter('"');
delimitedLineTokenizer.setNames(IrcRevisedData.getFields());

完整修改后的ItemReader配置代码 #

FlatFileItemReader<IrcRevisedData> reader = new FlatFileItemReader<>();

File inputDir = new File(ircReferenceData.getInputDir());
if(!inputDir.isDirectory() || inputDir.list().length <= 0)
    throw new RuntimeException("Either input directory is not present or its empty");

if(ircRevisedReferenceData.getFileName() == null){
    throw new RuntimeException("File not found");
}
Resource inputFile = new FileSystemResource(inputDir + File.separator + ircRevisedReferenceData.getFileName());
reader.setResource(inputFile);


DelimitedLineTokenizer delimitedLineTokenizer = new DelimitedLineTokenizer(",");
// 配置引号字符，处理字段内的逗号
delimitedLineTokenizer.setQuoteCharacter('"');
delimitedLineTokenizer.setNames(IrcRevisedData.getFields());

BeanWrapperFieldSetMapper<IrcRevisedData> beanWrapperFieldSetMapper = new BeanWrapperFieldSetMapper<>();
beanWrapperFieldSetMapper.setTargetType(IrcRevisedData.class);

DefaultLineMapper<IrcRevisedData> defaultLineMapper = new DefaultLineMapper<>();
defaultLineMapper.setLineTokenizer(delimitedLineTokenizer);
defaultLineMapper.setFieldSetMapper(beanWrapperFieldSetMapper);
reader.setRecordSeparatorPolicy(new DefaultRecordSeparatorPolicy());
reader.setLineMapper(defaultLineMapper);
reader.setLinesToSkip(3);
return reader;

原理说明 #

设置quoteCharacter('"')后，DelimitedLineTokenizer会识别被双引号包裹的内容为单个字段，忽略其中的逗号分隔符，这样"john,bill"就会被当作column2的完整值，不会被拆分成两列。

内容的提问来源于stack exchange，提问作者breakingcode