Visual Studio下C++代码时间测量异常问题排查

Great question! The inconsistent timing results you're seeing in Visual Studio 2015 (vs. stable results in GCC) are likely tied to a combination of compiler optimization heuristics, CPU behavior, and benchmarking pitfalls. Let's break down what's happening and how to fix it:

First, let's recap your setup #

Your code repeatedly copies elements from vec1 to vec2 in five identical loop blocks, expecting consistent timing. Here's your core code (formatted for clarity):

#include <iostream>
#include <chrono>
#include <vector>

int main() {
    std::srand(time(NULL));
    constexpr unsigned vectorSize = 1000u;
    constexpr unsigned loopCount = 1000000u;
    std::vector<int> vec1(vectorSize);
    std::vector<int> vec2(vectorSize);

    for (unsigned i = 0u; i < vectorSize; ++i) {
        vec1[i] = std::rand();
    }
    for (unsigned i = 0u; i < vectorSize; ++i) {
        vec2[i] = std::rand();
    }

    std::chrono::time_point<std::chrono::high_resolution_clock> start, end;
    long long ms;

    // Repeat identical copy loops 5 times
    for (int k = 0; k < 5; ++k) {
        start = std::chrono::high_resolution_clock::now();
        for (unsigned j = 0u; j < loopCount; ++j) {
            for (unsigned i = 0u; i < vec2.size(); ++i) {
                vec2[i] = vec1[i];
            }
        }
        end = std::chrono::high_resolution_clock::now();
        ms = std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
        std::cout << "Evaluation took: " << ms << " ms" << std::endl;
    }

    std::cout << "Press enter to exit..." << std::endl;
    std::cin.get();
    return 0;
}

Your VS2015 Release output shows puzzling timing swings:

Evaluation took: 425 ms
Evaluation took: 694 ms
Evaluation took: 462 ms
Evaluation took: 441 ms
Evaluation took: 710 ms

Why this happens in VS2015 #

1. Inconsistent Compiler Optimization Heuristics
   VS2015's optimizer has quirks with repeated identical code blocks. It might apply aggressive optimizations (like SIMD vectorization, loop unrolling) to some blocks but not others, based on internal heuristics. For example, it could flag the first block as a good candidate for vectorization, but skip the second due to perceived redundancy or resource limits—leading to drastically different runtime performance.

   Note: Your attempt to avoid SIMD by using vec2.size() won't work in Release mode. std::vector::size() is an inline function that returns a stored member variable, which the compiler optimizes to a constant (since your vector size never changes). So SIMD is still enabled, and the optimizer's decision to apply it can vary between blocks.

2. CPU Frequency Scaling & Cache Behavior
   Modern CPUs adjust their frequency dynamically. Your first run might catch the CPU in a higher Turbo Boost state, but after sustained load, the CPU might throttle back to base frequency for subsequent runs. Alternatively, after the first copy, vec2 matches vec1, so subsequent writes could trigger different cache write-back behavior, though this is less likely to cause such large swings.

3. Lack of Benchmark Warm-Up
   You're starting timing immediately without priming the CPU or cache. While your first run is fast, subsequent runs might be affected by background OS processes or CPU state changes that weren't present initially.

Fixes for Consistent Benchmarking #

To get reliable, repeatable results, try these steps:

• Wrap Core Logic in a Function
  Instead of copying the loop block five times, put the copy logic in a separate function and call it repeatedly. This ensures the compiler applies the same optimization strategy every time:

  void copyVec(const std::vector<int>& src, std::vector<int>& dest, unsigned loopCount) {
      for (unsigned j = 0u; j < loopCount; ++j) {
          for (unsigned i = 0u; i < dest.size(); ++i) {
              dest[i] = src[i];
          }
      }
  }
  
  // In main:
  for (int k = 0; k < 5; ++k) {
      start = std::chrono::high_resolution_clock::now();
      copyVec(vec1, vec2, loopCount);
      end = std::chrono::high_resolution_clock::now();
      // ... timing code ...
  }
  

• Add a Warm-Up Phase
  Run the copy function once (or a few times) before starting your actual measurements. This primes the cache and gets the CPU into a stable frequency state:

  // Warm-up
  copyVec(vec1, vec2, loopCount);
  
  // Actual measurements
  for (int k = 0; k < 5; ++k) {
      // ... timing code ...
  }
  

• Disable CPU Frequency Scaling
  For precise benchmarking, disable Turbo Boost and set your CPU to run at a fixed base frequency (via BIOS or Windows power settings). This eliminates frequency-related timing variance.

• Use More Precise Timing
  Instead of milliseconds, use std::chrono::microseconds or nanoseconds to capture smaller differences, then calculate averages over multiple runs to reduce noise.

• Explicitly Control SIMD Optimization
  If you want to force-disable SIMD (to test scalar performance), use the VS compiler flag /arch:IA32 (disables SSE/SSE2/AVX). Alternatively, if you want consistent SIMD, use /arch:AVX2 (if your CPU supports it) to enable advanced vectorization.

Final Notes #

GCC's optimizer tends to handle repeated code blocks more consistently than VS2015's, which is why you see stable results there. By structuring your benchmark properly and controlling for CPU/compiler variables, you'll get the reliable timing data you need for low-level performance testing.

内容的提问来源于stack exchange，提问作者Paweł Kozłowski