性能优化/替代方案：从数组头部移除x个元素

Great question—let’s dive into the most efficient ways to trim those arrays from the head, since fixing this is exactly what’ll slash your processing time from 20 seconds down to 2. The key here is avoiding slow, manual element shifting and leveraging JavaScript’s optimized native methods.

1. The Best-in-Class Solution: Array.prototype.slice() #

The fastest way to keep only the last 250 elements (and ditch the rest from the head) is using slice(). This native method is heavily optimized by JavaScript engines like V8, making it way faster than any handwritten loop for large arrays.

Here’s how to apply it to your data object:

const MAX_ALLOWED_ELEMENTS = 250;

// Loop through all array properties in your data object
for (const key in data) {
  const array = data[key];
  if (Array.isArray(array) && array.length > MAX_ALLOWED_ELEMENTS) {
    // Keep only the last 250 elements by slicing from the calculated start index
    data[key] = array.slice(array.length - MAX_ALLOWED_ELEMENTS);
  }
}

Why this works so well: #

• slice() creates a new array with only the elements you need (the last 250) in O(k) time where k is the number of elements to keep (250, in your case).
• Unlike methods that modify the original array (like splice), slice() avoids the overhead of shifting thousands of elements around. The engine handles the copy operation at a low level, which is far more efficient.

2. Alternative: Array.prototype.splice() (Less Optimal, But Still Better Than Shifting) #

If you prefer modifying the original array instead of creating a new one, you can use splice() to remove excess elements from the head. However, note that this is slightly slower than slice() for very large arrays because it has to shift the remaining elements in place.

Example usage:

const MAX_ALLOWED_ELEMENTS = 250;

for (const key in data) {
  const array = data[key];
  if (Array.isArray(array) && array.length > MAX_ALLOWED_ELEMENTS) {
    // Calculate how many elements to remove from the start
    const elementsToRemove = array.length - MAX_ALLOWED_ELEMENTS;
    array.splice(0, elementsToRemove);
  }
}

3. What You Must Avoid: Looping with shift() #

This is the mistake that’s almost certainly causing your 20-second runtime. Using shift() in a loop to remove elements from the head is catastrophically slow for large arrays:

// ❌ DO NOT USE THIS — EXTREMELY SLOW FOR LARGE ARRAYS
while (data.ids.length > MAX_ALLOWED_ELEMENTS) {
  data.ids.shift();
  data.names.shift();
  data.ages.shift();
  data.relations.shift();
}

Why this is terrible: #

Every shift() operation requires JavaScript to reindex every remaining element in the array (shifting them all left by one position). For an array of 70,000 elements, you’d run this operation 69,750 times—leading to O(n²) time complexity, which is exponentially slower than the O(k) methods above.

4. Bonus: Batch Processing for Continuous Data #

Since you’re adding data continuously, optimize further by trimming arrays only after batches of new data are added (instead of checking after every single addition). This reduces the number of trim operations you perform:

const MAX_ALLOWED_ELEMENTS = 250;

// Add new data in batches
function addBatchData(newIds, newNames, newAges, newRelations) {
  // Push all new elements at once
  data.ids.push(...newIds);
  data.names.push(...newNames);
  data.ages.push(...newAges);
  data.relations.push(...newRelations);

  // Trim all arrays in one go
  trimAllArrays(data);
}

function trimAllArrays(obj) {
  for (const key of Object.keys(obj)) {
    const array = obj[key];
    if (Array.isArray(array) && array.length > MAX_ALLOWED_ELEMENTS) {
      obj[key] = array.slice(array.length - MAX_ALLOWED_ELEMENTS);
    }
  }
}

Using slice() as shown here should immediately bring your runtime down to the 2-second range, as it eliminates the O(n²) overhead and leverages the engine’s optimized native code.

内容的提问来源于stack exchange，提问作者Tiago