基于多列过滤数组去除重复记录的实现方法

输入数组 #

let inputArr = [
    {
        "supplierId": "1",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 1",
        "partNumber": "4018992",
        "partDescription": "Description 1"
    },
    {
        "supplierId": "2",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 2",
        "partNumber": "4018992",
        "partDescription": "Description 2"
    },
    {
        "supplierId": "3",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 3",
        "partNumber": "4019030",
        "partDescription": "Description 3"
    },
    {
        "supplierId": "4",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 4",
        "partNumber": "4018291",
        "partDescription": "Description 4"
    },
    {
        "supplierId": "5",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 5",
        "partNumber": "4018992",
        "partDescription": "Description 5"
    }
]

需求 #

根据baseSupplierId和partNumber的组合值去重，若存在多个组合值相同的记录，仅保留第一条，移除其余重复项。

解决方案 #

用Array.reduce()方法遍历数组，结合对象记录已出现的组合，实现去重并保留原顺序：

// 记录已出现的baseSupplierId+partNumber组合
const seen = {};
const outputArr = inputArr.reduce((acc, curr) => {
    // 生成唯一标识组合的键
    const key = `${curr.baseSupplierId}_${curr.partNumber}`;
    if (!seen[key]) {
        seen[key] = true;
        acc.push(curr);
    }
    return acc;
}, []);

输出结果 #

let outputArr = [
    {
        "supplierId": "1",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 1",
        "partNumber": "4018992",
        "partDescription": "Description 1"
    },
    {
        "supplierId": "3",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 3",
        "partNumber": "4019030",
        "partDescription": "Description 3"
    },
    {
        "supplierId": "4",
        "baseSupplierId": "71277",
        "supplierName": "Supplier 4",
        "partNumber": "4018291",
        "partDescription": "Description 4"
    }
]

逻辑说明 #

• 用seen对象存储已处理的组合键，确保每个baseSupplierId+partNumber组合只被记录一次。
• 遍历数组时，若当前记录的组合未出现过，就将其加入结果数组并标记为已出现；若已出现则直接跳过。
• 该方法处理效率为O(n)，且严格保留原数组中重复组合的第一条记录。

内容的提问来源于stack exchange，提问作者Shubham Khandelwal