FastUtil中Long类型Set操作失效问题排查求助

问题根源 #

你代码中使用的ReferenceOpenHashSet<Long>是fastutil提供的引用哈希集合，它判断元素相等的逻辑是直接比较对象引用（==），而非调用Long的equals()方法。

Java对Long类型有缓存机制：-128到127之间的数值会复用同一个Long对象，所以小数值场景下==和equals()结果一致，能正常匹配；但超出这个范围的Long值会创建新对象，即使数值相同，引用也不相等，导致contains()方法无法识别匹配的元素，最终相似性计算返回0。

修复方案 #

改用fastutil专为基本类型long优化的LongOpenHashSet，它直接操作基本类型值，不存在引用相等性的问题，同时性能比包装类型集合更高效。

修改后的完整代码 #

package net.clementlevallois.functions.mapsofscience;

import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
import java.io.IOException;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;

public class JournalSimilaritiesComputer {

    static Long2ObjectOpenHashMap<LongOpenHashSet> journal2AuthorsMap = new Long2ObjectOpenHashMap<>();

    public static void main(String[] args) throws IOException {
        JournalSimilaritiesComputer computer = new JournalSimilaritiesComputer();
        computer.loadDataToMap();
        System.out.println("number of entries in the map: " + journal2AuthorsMap.size());
        computer.doubleLoopingThroughJournalIds();
    }

    private void loadDataToMap() throws IOException {
        // List<String> lines = List.of("1|0","2|0"); // 原正常场景
        List<String> lines = List.of("4210168784|3089954349","198793727|3089954349"); // 原失效场景
        
        lines.stream().forEach(this::processLine);
    }

    private void doubleLoopingThroughJournalIds() {        
        journal2AuthorsMap.keySet().longStream().forEach(journalIdA -> {
            LongOpenHashSet authorsOfJournalA = journal2AuthorsMap.get(journalIdA);

            journal2AuthorsMap.keySet().longStream().forEach(journalIdB -> {
                if (journalIdA == journalIdB) {
                    return;
                }
                LongOpenHashSet authorsOfJournalB = journal2AuthorsMap.get(journalIdB);
                Integer similarity = computeSimilarities(authorsOfJournalA, authorsOfJournalB);
                if (similarity > 0) {
                    String sim = journalIdA + "," + journalIdB + "," + similarity;
                    System.out.println("similarity detected! -> " + sim);
                }
            });
        });
    }

    private Integer computeSimilarities(LongOpenHashSet authorsOfJournalA, LongOpenHashSet authorsOfJournalB) {
        // 直接用基本类型long遍历，避免包装类型问题
        authorsOfJournalA.longStream().forEach(entry -> {
            if (authorsOfJournalB.contains(entry)){
                System.out.println("stop"); // 现在会触发
            }
        });
        Set<Long> commonElements = authorsOfJournalA.longStream()
                .boxed()
                .filter(authorsOfJournalB::contains)
                .collect(Collectors.toSet());
        return commonElements.size(); // 现在返回1，符合预期
    }

    private void processLine(String line) {
        String fields[] = line.split("\\|");
        if (fields.length < 2) {
            return;
        }
        long journalIdAsLong = Long.parseLong(fields[0]);
        String authorIds[] = fields[1].split(",");
        
        LongOpenHashSet setOfAuthorsForThisJournal = journal2AuthorsMap.getOrDefault(journalIdAsLong, new LongOpenHashSet());
        for (String authorId : authorIds) {
            try {
                long authorIdLong = Long.parseLong(authorId);
                setOfAuthorsForThisJournal.add(authorIdLong);
            } catch (NumberFormatException e) {
                System.out.println("error with author id, not long: " + authorId);
            }
        }
        journal2AuthorsMap.put(journalIdAsLong, setOfAuthorsForThisJournal);
    }

}

关键修改点说明 #

1. 替换集合类型：将ReferenceOpenHashSet<Long>改为LongOpenHashSet，直接操作基本类型long，彻底规避包装类型的引用相等性问题。
2. 简化集合初始化：processLine中直接用new LongOpenHashSet()作为getOrDefault的默认值，代码更简洁。
3. 优化遍历方式：使用longStream()直接遍历基本类型值，避免不必要的自动装箱。

内容的提问来源于stack exchange，提问作者seinecle