为何AVX-512处理数组时，批次大小8比7或9性能显著更低？

问题背景 #

这段代码将int64_t类型的输入数组按指定批次大小转换为double类型的输出数组：当剩余元素≥8时，使用AVX-512指令集批量处理以提升性能；否则回退到标量实现。为避免编译器合并循环，每个批次处理后通过asm volatile内存屏障强制刷新输出数据。

在Intel(R) Xeon(R) Gold 5220R CPU上，通过以下命令编译执行：

clang++ -Wall -Wextra -march=cascadelake -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vnni -O3 minimal.cpp -o minimal

测试结果却与预期相悖：批次大小为8时性能显著低于7或9，而理论上8应完美适配AVX-512的64字节处理能力，性能最优。

测试代码 #

#include <immintrin.h>

#include <algorithm>
#include <ctime>
#include <iostream>
#include <numeric>
#include <vector>

#define NUMBER_OF_TUPLES 134'217'728UL

void transform(std::vector<int64_t>* input, std::vector<double>* output, size_t batch_size) {
  for (size_t startOfBatch = 0; startOfBatch < NUMBER_OF_TUPLES; startOfBatch += batch_size) {
    size_t endOfBatch = std::min(startOfBatch + batch_size, NUMBER_OF_TUPLES);

    for (size_t idx = startOfBatch; idx < endOfBatch;) {
      if (endOfBatch - idx >= 8) {
        auto _loaded = _mm512_loadu_epi64(&(*input)[idx]);
        auto _converted = _mm512_cvtepu64_pd(_loaded);

        _mm512_storeu_epi64(&(*output)[idx], _converted);
        idx += 8;
      } else {
        (*output)[idx] = static_cast<double>((*input)[idx]);
        idx++;
      }
    }

    asm volatile("" : : "r,m"(output->data()) : "memory");
  }
}

void do_benchmark(size_t batch_size) {
  std::vector<int64_t> input(NUMBER_OF_TUPLES);
  std::vector<double> output(NUMBER_OF_TUPLES);

  std::iota(input.begin(), input.end(), 0);

  auto t = std::clock();
  transform(&input, &output, batch_size);
  auto elapsed = std::clock() - t;

  std::cout << "Elapsed time for a batch size of " << batch_size << ": " << elapsed << std::endl;
}

int main() {
  do_benchmark(7UL);
  do_benchmark(8UL);
  do_benchmark(9UL);
}

测试结果 #

Elapsed time for a batch size of 7: 204007
Elapsed time for a batch size of 8: 237600
Elapsed time for a batch size of 9: 209838

Perf缓存缺失数据 #

Batch Size 7 #

Performance counter stats for process id '653468':

     6,894,467,363      L1-dcache-loads                                               (44.43%)
     1,647,244,371      L1-dcache-load-misses     #   23.89% of all L1-dcache accesses  (44.43%)
     7,548,224,648      L1-dcache-stores                                              (44.43%)
         6,726,036      L2-loads                                                      (44.43%)
         3,766,847      L2-loads-misses           #   56.61% of all LL-cache accesses  (44.46%)
         6,171,407      L2-loads-stores                                               (44.45%)
         6,764,242      LLC-loads                                                     (44.46%)
         4,548,106      LLC-loads-misses          #   68.35% of all LL-cache accesses  (44.46%)
         6,954,088      LLC-loads-stores                                              (44.45%)

Batch Size 8 #

Performance counter stats for process id '654880':

     1,009,889,247      L1-dcache-loads                                               (44.41%)
     1,413,152,123      L1-dcache-load-misses     #  139.93% of all L1-dcache accesses  (44.45%)
     1,528,453,525      L1-dcache-stores                                              (44.48%)
       158,053,929      L2-loads                                                      (44.51%)
       155,407,942      L2-loads-misses           #   98.18% of all LL-cache accesses  (44.50%)
       158,335,431      L2-loads-stores                                               (44.46%)
       158,349,901      LLC-loads                                                     (44.42%)
       155,902,630      LLC-loads-misses          #   98.49% of all LL-cache accesses  (44.39%)
       158,447,095      LLC-loads-stores                                              (44.39%)

      11.011153400 seconds time elapsed

Batch Size 9 #

Performance counter stats for process id '656032':

     1,766,679,021      L1-dcache-loads                                               (44.38%)
     1,600,639,108      L1-dcache-load-misses     #   90.60% of all L1-dcache accesses  (44.42%)
     2,233,035,727      L1-dcache-stores                                              (44.46%)
       138,071,488      L2-loads                                                      (44.49%)
       136,132,162      L2-loads-misses           #   98.51% of all LL-cache accesses  (44.52%)
       138,020,805      L2-loads-stores                                               (44.49%)
       138,522,404      LLC-loads                                                     (44.45%)
       135,902,197      LLC-loads-misses          #   98.35% of all LL-cache accesses  (44.42%)
       138,122,462      LLC-loads-stores                                              (44.38%)

原因分析 #

结合perf数据和代码逻辑，性能反常的核心原因如下：

1. 内存屏障强制缓存刷新
   每个批次后的asm volatile内存屏障会强制编译器将输出数据写回主存，同时标记缓存行失效。这意味着后续批次的内存访问无法复用缓存中的数据，必须重新从主存加载。

2. 缓存行被持续驱逐
   批次大小为8时，每个批次刚好处理一个64字节的缓存行（输入、输出各一个）。连续的写操作会持续占用新的缓存行，输入的缓存行在处理完成后会被快速挤出缓存，导致下一个批次的输入必须从主存重新加载，直接造成极高的L1、LLC缓存缺失率（批次8的LLC缺失率高达98.49%）。

3. 预取器失效
   严格的固定批次大小导致内存访问模式过于规律，但内存屏障打断了CPU预取器的连续预测逻辑，无法提前加载后续的缓存行，进一步加剧了内存访问延迟。

4. 无分支流水线的停顿
   批次8时内层循环完全无分支，流水线持续执行AVX-512指令，但内存访问的高延迟导致流水线频繁停顿。而批次7和9存在标量分支，CPU反而能在分支预测的间隙进行预取或其他操作，缓解了内存延迟的影响。

内容的提问来源于stack exchange，提问作者InvisibleShadowGhost