如何从数组中获取唯一值？去重后数据未变化问题排查

Hey, I’ve dealt with this exact kind of problem when working with DataTables, so let’s break down why your deduplication isn’t hitting the mark and how to fix it.

1. Most Likely Culprit: Your years Array is Nested (2D Array) #

When you use DataTables' columns(2).data().toArray(), it can sometimes return a 2D nested array instead of a flat 1D array. That means your years might look like [[2016], [2016], [2015], ...] instead of [2016, 2016, 2015, ...].

Since each element is a separate array object (even if they contain the same value), both Set and Lodash's _.uniq can’t deduplicate them—they treat each array reference as unique.

First, confirm this by logging the actual structure of years:

console.log("Raw years array:", JSON.stringify(years));

If it’s a nested array, fix it by flattening first:

// Option 1: Use Array.flat() to flatten the nested array
var years = this.api().columns(2).data().toArray().flat().sort();

// Option 2: Use DataTables' pluck() to get a flat array directly
var years = this.api().columns(2).data().pluck(0).toArray().sort();

2. Mixed Data Types (Strings vs. Numbers) #

Another common issue is having a mix of string and numeric values in your array. For example, some elements might be "2016" (string) and others 2016 (number). Set and _.uniq consider these different values because their types don’t match.

Check the types of your elements with this quick log:

years.forEach(year => console.log(`Type: ${typeof year}, Value: ${year}`));

Fix this by converting all elements to the same type before deduplication:

// Convert all to numbers
var years = this.api().columns(2).data().toArray().map(Number).sort();

// OR convert all to strings
// var years = this.api().columns(2).data().toArray().map(String).sort();

3. Better Way to Verify Results #

Instead of concatenating the array to a string (which hides the actual structure), log the arrays directly to see what’s really going on:

console.log("Before deduplication:", years); 
var duplicateFreeYears = Array.from(new Set(years)); 
const uniqueTopicList = _.uniq(years); 
console.log("After Set deduplication:", duplicateFreeYears); 
console.log("After Lodash deduplication:", uniqueTopicList);

This will show you the true shape of your arrays and confirm if the deduplication worked.

内容的提问来源于stack exchange，提问作者GettingStarted