JSObject存入2^23条数据后性能骤降的底层原因分析与性能排查咨询

Great question! The dramatic performance drop you're seeing after 2^23 items is directly tied to how V8 handles object property storage under the hood, combined with predictable key patterns causing severe hash collisions. Let's break this down and cover how to validate and profile the issue:

Core Root Cause #

1. V8's Fast-to-Slow Property Mode Switch #

V8 optimizes plain JavaScript objects (like the one created with Object.create(null)) using two storage modes:

• Fast Properties: By default, V8 uses a contiguous FixedArray to store property values, with metadata (like property names and offsets) managed via a hidden class (aka Map). This enables O(1) average time for property writes/reads.
• Slow Properties (Dictionary Mode): V8 has a hard limit on the number of fast properties an object can hold: kMaxFastProperties = 2^23 (8,388,608). Once your object exceeds this threshold, V8 switches it to dictionary mode, where properties are stored in a hash table instead of a contiguous array.

2. Severe Hash Collisions from Predictable Keys #

Your keys follow a strict pattern: prop_0, prop_1, prop_2, ..., prop_N. V8's string hashing function produces highly correlated hash values for these sequential keys, leading to massive hash collisions in the dictionary's hash table.

Instead of average O(1) writes, each new property insertion now has to traverse long collision chains to find an empty slot—degrading to O(n) time per insertion. This is why you see write times jump from 0.001ms to 1.5s per iteration after crossing the 2^23 threshold.

When you use random hash-based keys, collisions become extremely rare, so dictionary mode operations stay fast (average O(1)), which explains why that workaround works.

How to Validate & Profile the Issue #

1. Confirm Mode Switch with %DebugPrint #

You're already using %DebugPrint(a)—look for these clues in the output:

• Fast Mode: You'll see properties: 0x... [FixedArray] and the map entry will not have is_dictionary_map: true.
• Slow/Dictionary Mode: You'll see properties: 0x... [Dictionary] and is_dictionary_map: true in the map details.

After crossing 2^23 items, the output will clearly show the object has switched to dictionary mode.

2. Profile with Chrome DevTools #

Chrome DevTools lets you visualize exactly where time is being spent:

1. Run your script with the inspect flag:

   node --allow-natives-syntax --inspect-brk file.js
   

2. Open Chrome and navigate to chrome://inspect—click "Inspect" next to your Node.js process.
3. Go to the Performance tab, click "Record", and let the script run until it hits the slow phase.
4. Stop recording and analyze the flame graph: You'll see overwhelming time spent in V8's internal dictionary/hash table functions (e.g., Dictionary::Add, HashTable::FindInsertionEntry).

3. Use V8's Built-in CPU Profiler #

For low-level V8-specific data, use the built-in profiler:

1. Run your script with the profiler enabled:

   node --allow-natives-syntax --prof file.js
   

2. A file named isolate-0x...-v8.log will be generated. Process it with:

   node --prof-process isolate-0x...-v8.log > profile.txt
   

3. Open profile.txt and look for high CPU time in functions related to hash table operations (e.g., HashTable* or Dictionary* functions). This will confirm that collision resolution is dominating the slowdown.

4. Validate Hash Collision Hypothesis #

Modify your script to use random keys (mimicking your "hash key" workaround) and compare performance:

function foo() {
    const result = Object.create(null);
    for (let i = 0; i < 25_000_000; i += 1) {
        // Random key instead of sequential pattern
        const randomKey = `hash_${Math.random().toString(36).slice(2)}`;
        console.time(`Prop ${i}`);
        result[randomKey] = i + (Math.random() * 25_000_000);
        console.timeEnd(`Prop ${i}`);
    }
    return result;
}

You'll see that performance stays fast even past the 2^23 threshold, confirming the key pattern is the culprit.

Optimization Recommendations #

If you need to store millions of key-value pairs:

1. Use Arrays for Sequential Keys: Since your keys are effectively sequential numbers, an array is far more efficient (V8 optimizes arrays heavily for contiguous storage):

   function foo() {
       const result = [];
       for (let i = 0; i < 25_000_000; i += 1) {
           console.time(`Prop ${i}`);
           result[i] = i + (Math.random() * 25_000_000);
           console.timeEnd(`Prop ${i}`);
       }
       return result;
   }
   

2. Use Map Instead of Plain Objects: Map is designed for large collections and has better hash collision handling than plain objects in dictionary mode. It also avoids the fast-to-slow mode switch entirely.

备注：内容来源于stack exchange，提问作者Samet Aylak