基于Fortran代码的缓存测试疑问：未观测到Gflops下降的原因

Let’s walk through why you might not be seeing the expected GFLOPS drop when your matrix size exceeds the L1 data cache limit, and break down how to adjust your test to observe the behavior you’re looking for.

Key Misconceptions in Cache Usage Estimation #

Your initial calculation of 3*N*N*8 bytes for total data size is technically correct for the full matrices, but cache doesn’t work by storing all three matrices at once—especially not when your access pattern is optimized for Fortran’s column-major storage. Here’s why that matters:

1. Column-Major Access Maximizes Cache Reuse & Hits #

Fortran stores matrices column-first, so when you iterate over columns first (e.g., the loop below), you’re accessing contiguous memory blocks:

do j = 1, N
  do i = 1, N
    A(i,j) = A(i,j) + B(i,j) + C(i,j)
  enddo
enddo

Each 64-byte cache line holds 8 real*8 elements (64/8=8), so loading one cache line lets you process 8 elements across all three matrices. Instead of needing to fit all three matrices in L1, you only need to fit small, contiguous chunks of each matrix’s column at a time—for example, one cache line from each matrix uses just 3*64 = 192 bytes, way under your 32KB L1d limit. Even as N grows, the CPU’s prefetchers automatically load the next set of contiguous cache lines before you need them, keeping cache hits high.

2. Cache Hierarchy Buffers Overflows #

Your system has L2 (256KB) and L3 (3MB) caches sitting between L1 and main memory. If your total data size exceeds L1 but fits in L2, the performance drop will be minimal—L2 has much lower latency than main memory, so GFLOPS won’t plummet like it would when hitting main memory. You might not have tested N large enough to exceed L2 (or even L3) yet:

• L1d can hold 32768 / 8 = 4096 real*8 elements
• L2 can hold 262144 /8 = 32768 real*8 elements
• L3 can hold 3145728 /8 = 393216 real*8 elements

For three matrices, the threshold where total data exceeds L2 is when 3*N² > 32768 → N ≈ 104. Exceeding L3 would be 3*N²>393216 → N≈363. If your test only went up to N values that fit in L2, you wouldn’t see a sharp GFLOPS drop.

3. Calculation-to-Memory Ratio Limits Bottlenecks #

Your operation (A = A+B+C) does 2 floating-point operations per element but moves 4*8=32 bytes of data (read A, B, C; write A). That’s a low computation-to-memory ratio (1 FLOP per 16 bytes). Modern CPUs have high memory bandwidth, so even with some cache misses, the memory subsystem can keep up with the CPU’s compute rate until cache misses become extreme. You won’t see a GFLOPS drop until memory bandwidth is saturated.

4. Loop Repetition & Cache Warm-Up #

If you’re repeating the loop multiple times to average timing, the first iteration may load data into cache, but subsequent iterations might reuse cached data even if N exceeds L1—thanks to prefetchers keeping data flowing into cache before it’s evicted. This can mask cache misses in your average timing.

Steps to Observe the Expected GFLOPS Drop #

To see the cache performance cliff you’re expecting, try these adjustments:

• Expand your N range: Test from small values (N=10) up to N=400+ (to exceed L3). You should see three distinct drops: a subtle one when exceeding L1, a larger drop at L2, and a sharp drop when exceeding L3 and hitting main memory.
• Compare row-wise vs column-wise access: Implement a row-first loop (swap the order of i and j loops). This will access non-contiguous memory (skipping N elements each time), leading to massive cache misses even for small N. You’ll see a dramatic GFLOPS drop here, highlighting the difference between optimal and poor cache usage.
• Measure cache misses directly: Use a performance profiling tool (like perf on Linux) to count L1d, L2, and L3 cache misses as you increase N. This lets you correlate miss rates with GFLOPS changes, making it clear when each cache level is overwhelmed.
• Adjust loop unrolling: Try manually unrolling your inner loop to increase computation density. This reduces the number of memory accesses relative to compute operations, making cache misses more impactful on GFLOPS.

内容的提问来源于stack exchange，提问作者j.vaquero