Turing架构GPU（算力7.5）中Warp线程全局内存访问机制问询

Great question—let’s break this down clearly, since global memory handling has key differences from shared memory (even though there’s some overlap in broadcast behavior). Turing’s memory architecture is optimized to handle these warp-level access patterns efficiently, so let’s tackle your two examples one by one:

Example 1: First 16 threads access one 4-byte word, next 16 access the adjacent 4-byte word #

First, a quick baseline: Turing’s global memory uses memory transactions (typically 32-byte aligned blocks for most common accesses) to fetch data. When threads in a warp access addresses within the same 32-byte block, the hardware merges those requests into a single transaction.

In your case:

• The first 16 threads target address A (a 4-byte word), and the next 16 target A+4 (the immediate next 4-byte word). Both addresses fall within the same 32-byte aligned block (since 32 bytes = 8 consecutive 4-byte words).
• The GPU will issue a single 32-byte memory transaction to read the entire block containing A and A+4.
• For the 16 threads requesting A, the hardware broadcasts the value at A to all 16 threads (no extra overhead here).
• For the 16 threads requesting A+4, it does the same: broadcasts the value at A+4 to those 16 threads.

No serialization happens here—this is a fully coalesced, efficient access. The entire warp’s request is handled in one memory transaction, with broadcast taking care of the multiple threads targeting the same address within the block.

Example 2: Entire warp accesses the same 4-byte word #

This is even more straightforward. When all 32 threads in a warp request the exact same 4-byte global memory address:

• The GPU issues a single memory transaction (it optimizes this to only fetch the necessary 4 bytes, avoiding redundant reads of unused memory in the aligned block).
• That single fetched value is broadcast to all 32 threads in the warp simultaneously.

This is the most efficient case for this access pattern—no redundant memory operations, no overhead, just one fetch and a warp-wide broadcast.

Key Difference from Shared Memory #

You mentioned shared memory bank conflicts, so it’s worth clarifying:

• Shared memory bank conflicts occur because the SM’s shared memory is split into 32 banks, each serving one access per cycle. Multiple threads accessing different addresses in the same bank cause conflicts, though shared memory also broadcasts a single address to all warp threads if they target it.
• Global memory doesn’t have "bank conflicts" in the same sense. Its accesses are handled by the GPU’s memory controllers, which natively support broadcasting a single address to multiple threads in a warp. So same-address accesses are always efficient, regardless of how many threads are requesting the value.

内容的提问来源于stack exchange，提问作者haykoandri