Post-Pascal架构下GPU与主机内存均耗尽时，如何让Unified Memory利用磁盘交换空间？

Great question—this is a common point of confusion with CUDA Unified Memory (UM) because its behavior doesn't perfectly align with standard system virtual memory by default. Let's break down why your experiment didn't use swap, and how to fix it.

Why Your UM Test Didn't Use Swap #

Post-Pascal GPUs do support oversubscribing UM beyond GPU/host memory, but NVIDIA's CUDA driver intentionally avoids swapping UM pages to disk by default. Here's why:

• GPU access to disk-based swap is extremely slow (orders of magnitude slower than host memory), so the driver prioritizes keeping UM pages in physical host/GPU memory to avoid catastrophic performance hits.
• Linux's OOM-killer treats CUDA-managed memory as "unreclaimable" by default. When host physical memory runs out, the kernel kills the process instead of trying to swap UM pages, since it doesn't recognize them as eligible for swapping.

In contrast, standard malloc memory is fully managed by the system, so it follows normal virtual memory rules and swaps to disk automatically.

How to Enable UM Swap to Disk #

To get UM to use disk swap when physical memory is exhausted, you need to adjust CUDA's memory policies and let the system know these pages are eligible for swapping. Here's how:

1. Use Environment Variables to Unlock Swap Eligibility #

Before running your program, set these environment variables to tell CUDA not to lock UM pages in physical memory:

# Allow the system to manage UM pages like regular virtual memory (including swap)
export CUDA_MANAGED_FORCE_ALLOC_HOST=0
# Ensure UM uses unified addressing tied to the system's virtual memory pool
export CUDA_MANAGED_UNIFIED_ADDRESSING=1

2. Add CUDA API Calls to Explicitly Allow Swapping #

Modify your code after cudaMallocManaged to advise the driver that these pages can be swapped, and set up proper access rules:

cudaMallocManaged(&x, N * sizeof(float));
cudaMallocManaged(&y, N * sizeof(float));

// Set preferred location to CPU (system memory), allowing the OS to swap these pages
cudaMemAdvise(x, N * sizeof(float), cudaMemAdviseSetPreferredLocation, cudaCpuDeviceId);
cudaMemAdvise(y, N * sizeof(float), cudaMemAdviseSetPreferredLocation, cudaCpuDeviceId);

// Notify the driver that the GPU will access this memory (triggers on-demand paging)
int deviceId = 0; // Adjust to your GPU's device ID
cudaMemAdvise(x, N * sizeof(float), cudaMemAdviseSetAccessedBy, deviceId);
cudaMemAdvise(y, N * sizeof(float), cudaMemAdviseSetAccessedBy, deviceId);

3. Tweak System Swap Behavior (Optional) #

If the OOM-killer still terminates your process, increase the system's swap tendency by adjusting the swappiness value (temporary setting, resets on reboot):

echo 100 > /proc/sys/vm/swappiness

A higher swappiness value makes the kernel more likely to use swap instead of killing processes. Use this only in test environments, as it can hurt overall system performance.

4. Verify the Fix #

Run your modified program and monitor swap usage with swapon --show or htop. You should see swap space being consumed, and the program won't be killed by the OOM-killer (though performance will be very slow—this is expected when accessing disk from the GPU).

Critical Notes #

• Performance Cost: GPU access to disk swap is prohibitively slow for most workloads. This is only useful for edge cases where you absolutely need to exceed physical memory limits.
• Architecture Support: This only works on Post-Pascal GPUs and newer. Older architectures (Kepler/Maxwell) don't support UM oversubscription to disk.
• Swap Configuration: Ensure your swap file/partition is properly sized and mounted (you already did this in your experiment, which is good).

内容的提问来源于stack exchange，提问作者66RING