Java实现grep时使用Stream的StringBuilder Collector程序挂起
Java Stream实现grep功能打包后collect方法挂起问题
我在入职培训项目中尝试用Java Stream实现grep功能,代码在IntelliJ中运行正常,但通过mvn clean package打包后,调用collect方法时程序出现挂起(省略了getter/setter及其他部分方法)。
JavaGrep类代码
class JavaGrep { public void process() throws IOException { Stream<Path> paths = listFiles(this.rootPath); logger.debug(this.outFile); paths.map( Path::toFile ).forEach( this::readLargeFile ); } public Stream<Path> listFiles(String rootDir) { try { return Files.walk(Paths.get(rootDir)).filter( path -> !Files.isDirectory(path) ); } catch (IOException e) { throw new RuntimeException("Error: IO Exception while reading root directory", e); } } void readLargeFile(File file) { try(FileInputStream fis = new FileInputStream(file);) { byte[] buffer = new byte[CHUNK_SIZE]; // Buffer to hold 20MB chunks int counter = 1; long fileSize = Files.size(file.toPath()); while (fis.read(buffer)!=-1) { logger.debug("Progress reading {} : {}", file.toPath(), Math.min((float) counter * (float)CHUNK_SIZE / (float)fileSize * 100.0, 100.0)); counter++; String s = new String(buffer, StandardCharsets.ISO_8859_1); Stream<String> lines = Arrays.stream(s.split("\n")); String chunkContent = lines.filter( this::containsPattern ).collect( new StringBuilderCollector() ); logger.debug("writing to file"); this.writeToFile(chunkContent); } } catch (IOException e) { throw new RuntimeException("Error while reading file " + file.getPath(), e); } } public void writeToFile(String lines) { Path outFilePath = Paths.get(this.outFile); try( FileChannel fileChannel = FileChannel.open(outFilePath, StandardOpenOption.APPEND, StandardOpenOption.CREATE); FileLock lock = fileChannel.lock() ) { Files.write(outFilePath, lines.getBytes(StandardCharsets.ISO_8859_1), StandardOpenOption.APPEND); } catch (IOException e) { logger.error("error while opening atomic file channel for {}", outFilePath, e); } } }
自定义Collector类代码
public class StringBuilderCollector implements Collector<String,StringBuilder,String> { StringBuilderCollector() { } @Override public Supplier<StringBuilder> supplier() { return StringBuilder::new; } @Override public BiConsumer<StringBuilder, String> accumulator() { return (sb, string) -> { sb.append(string); sb.append('\n'); }; } @Override public BinaryOperator<StringBuilder> combiner() { return (sb1, sb2) -> { sb1.append(sb2); return sb1; }; } @Override public Function<StringBuilder, String> finisher() { return StringBuilder::toString; } @Override public Set<Characteristics> characteristics() { return Collections.emptySet(); } }
已尝试的操作
- 若输出文件不存在则创建它
- 打印/日志记录(accumulator工作正常,能输出所有预期行,因此过滤部分应该没问题,但combiner内的代码似乎从未运行)
- 将Files.walk放入try()块,但这引发了非法状态异常。ChatGPT表示调用collect时资源会自动关闭,因此我认为这不是问题
内容的提问来源于stack exchange,提问作者qwerty_99
相关产品推荐
相关产品推荐

