Java并行流reduce为何未为每个线程创建独立累加器实例?
我是一名资深Java开发者,正在系统学习Streams。在测试reduce函数与StringBuffer/StringBuilder的结合使用时,发现了一处难以理解的行为:
- 串行流的reduce示例符合预期:
System.out.println(List.of("a", "b", "c", "d").stream().reduce(new StringBuilder(), (s, e)->s.append(e) , (sb1, sb2)->sb1==sb2?sb1:sb1.append(sb2)));
改用并行流并使用线程安全的StringBuffer时,发现combiner始终接收同一个StringBuffer对象,即使增加数据量也是如此。
尝试使用Supplier创建累加器并添加计数逻辑后,运行结果仍显示始终使用同一个实例:
var list = List.of("a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"); var counter = new HashMap<String, Integer>(); Supplier<StringBuffer> supplier = ()->{ var s = new StringBuffer(); s.append(counter.merge("c", 1, (a, b)->a+b)); System.out.printf("new:%s%n", s); return s; }; System.out.println(list.parallelStream().reduce(supplier.get(), (s, e)->s.append(e) , (sb1, sb2)->{System.out.printf("called with[%s] and [%s]%n", sb1, sb2);return sb1==sb2?sb1:sb1.append(sb2);}));
输出显示combiner的左右操作数常为同一对象:
called with[1qsr] and [1qsr] called with[1qsr] and [1qsr] called with[1qsrhi] and [1qsrhi] called with[1qsrhigno] and [1qsrhignopuv] called with[1qsrhignopuvlm] and [1qsrhignopuvlm] called with[1qsrhignopuvlmkjt] and [1qsrhignopuvlmkjt] called with[1qsrhignopuvlmkjtxwzy] and [1qsrhignopuvlmkjtxwzydfe] called with[1qsrhignopuvlmkjtxwzydfebc] and [1qsrhignopuvlmkjtxwzydfebc] called with[1qsrhignopuvlmkjtxwzydfebc] and [1qsrhignopuvlmkjtxwzydfebc] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca] called with[1qsrhignopuvlmkjtxwzydfebca] and [1qsrhignopuvlmkjtxwzydfebca]
问题根源
你犯了一个关键的重载调用错误:你使用的是reduce(T identity, BiFunction<T,T,T> accumulator, BinaryOperator<T> combiner)版本,而非带Supplier的reduce(Supplier<T> supplier, BiFunction<T,T,T> accumulator, BinaryOperator<T> combiner)版本。
你代码里的supplier.get()是直接生成一个StringBuffer实例作为identity参数传入,这意味着不管并行流怎么拆分任务,所有线程都会共享这同一个可变的StringBuffer实例——因为identity是全局初始值,并行流会复用它来累积所有线程的结果,自然combiner拿到的都是同一个对象。
正确用法
要让并行流为每个线程创建独立累加器,必须直接传入Supplier对象,而不是调用get()生成的实例:
var list = List.of("a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"); var counter = new HashMap<String, Integer>(); Supplier<StringBuffer> supplier = ()->{ var s = new StringBuffer(); s.append(counter.merge("c", 1, (a, b)->a+b)); System.out.printf("new:%s%n", s); return s; }; // 直接传supplier,而不是supplier.get() System.out.println(list.parallelStream().reduce(supplier, (s, e)->s.append(e) , (sb1, sb2)->{System.out.printf("called with[%s] and [%s]%n", sb1, sb2);return sb1.append(sb2);}));
运行这段代码,你会看到new:%s的输出多次出现,说明Supplier被调用了多次——也就是为不同的线程创建了独立的StringBuffer累加器,combiner也会接收到不同的实例进行合并。
补充说明
即使使用正确的Supplier版本,JVM也不会强制为每个线程都创建累加器——它会根据流的大小、并行度、任务拆分策略等因素动态决定创建多少个累加器,但至少会在需要并行处理时生成多个独立实例,而非始终复用同一个。
另外,注意StringBuilder是非线程安全的,在并行流的Supplier版本中使用是安全的,因为每个线程持有自己的实例;但如果用错成identity版本,会出现线程安全问题(即使StringBuffer线程安全,这种用法也不符合并行流reduce的设计意图)。
内容的提问来源于stack exchange,提问作者Eliezer Garza

