RTX 4090上每个CPU线程创建CUDA上下文占400MB，如何降低内存占用？

AI 小北

2026-6-16

问题描述

我正在开发一款利用NVIDIA解码能力的应用，需处理18路视频流，每路对应一个独立CPU线程，通过cuCtxCreate()为每个线程创建新CUDA上下文。在NVIDIA GeForce RTX 4090上，每个CUDA上下文占用约400MB GPU内存，远高于之前使用NVIDIA Quadro P4000时的水平。请问是否有方法降低每个CUDA上下文的GPU内存占用？

为排除应用中其他进程的影响，我编写了测试代码，创建多线程并为每个线程创建CUDA上下文，结果显示RTX 4090上每个线程仍占用约400MB GPU内存。测试代码如下：

#include <iostream>
#include <thread>
#include <vector>
#include <cuda.h>
#include <cuda_runtime.h>

static size_t previous_ocupancy = 0;

void threadFunction(int threadID) {

    size_t free_mem, total_mem = 0;

    int GPU_device = 0;
    CUdevice device;
    CUresult res = cuDeviceGet(&device, GPU_device);
    if (res != CUDA_SUCCESS) {
        std::cerr << "Failed to get device: " << GPU_device << std::endl;
        return;
    }

    cudaMemGetInfo(&free_mem, &total_mem);
    std::cout << ">>>cuDeviceGet: GPU FREE MEMORY: " << free_mem/1000000000.0 << ", Reserved: " << (total_mem-free_mem)/1000000000.0 << std::endl;

    std::this_thread::sleep_for(std::chrono::seconds(threadID));


    if (previous_ocupancy == 0) {
        previous_ocupancy = free_mem;
    }

    // Create CUDA context
    CUcontext g_oContext;
    res = cuCtxCreate(&g_oContext, CU_CTX_SCHED_BLOCKING_SYNC, device);
    if (res != CUDA_SUCCESS) {
        std::cerr << "Failed to create context." << std::endl;
        return;
    }

    cudaMemGetInfo(&free_mem, &total_mem);
    std::cout << ">>>cuCtxCreate: GPU FREE MEMORY: " << free_mem/1000000000.0
              << ", Reserved: " << (total_mem-free_mem)/1000000000.0
              << ", CTX memory: " << (previous_ocupancy-free_mem)/1000000.0 << " MiB" << std::endl;

    previous_ocupancy = free_mem;

    std::cout << "Thread " << threadID << " finished.\n";


    // Wait and start destroying CTX
    std::this_thread::sleep_for(std::chrono::seconds(40 + threadID));


    res = cuCtxDestroy(g_oContext);
    if (res != CUDA_SUCCESS) {
        std::cerr << "Failed to destroy context." << std::endl;
        return;
    }

    cudaMemGetInfo(&free_mem, &total_mem);
    std::cout << ">>>cuCtxDestroy: GPU FREE MEMORY: " << free_mem/1000000000.0
              << ", Reserved: " << (total_mem-free_mem)/1000000000.0 << std::endl;
}

int main() {

    int numThreads = 18; 

    std::vector<std::thread> threads;

    cuInit(0);

    // Create and start threads
    std::cout << "All threads START.\n";
    for (int i = 0; i < numThreads; ++i) {
        threads.push_back(std::thread(threadFunction, i));
    }

    // Wait for all threads to complete
    for (auto &th : threads) {
        th.join();
    }

    std::cout << "All threads have finished.\n";
    return 0;
}

优化方案

复用CUDA上下文而非逐个创建：无需为每个线程单独创建上下文，可通过cuCtxSetCurrent()在不同线程间切换同一个上下文。例如采用线程池模式，让多个解码任务复用少量上下文，从根源上减少上下文数量。
启用延迟初始化标志：创建上下文时添加CU_CTX_DEFER_INIT标志，上下文仅在实际执行CUDA操作时完成初始化，避免提前占用内存。修改创建代码为：
```
res = cuCtxCreate(&g_oContext, CU_CTX_SCHED_BLOCKING_SYNC | CU_CTX_DEFER_INIT, device);
```
禁用不必要的硬件组件初始化：RTX 4090默认会加载Tensor Core、RT Core相关组件，若解码任务不需要这些功能，可通过环境变量CUDA_DISABLE_TENSOR_CORE=1和CUDA_DISABLE_RT_CORE=1禁用，降低上下文内存占用。
调整上下文内存限制：通过cuCtxSetLimit()限制上下文的堆内存大小，在创建上下文后、执行内存操作前调用：
```
size_t heap_limit = 64 * 1024 * 1024; // 64MB
cuCtxSetLimit(CU_LIMIT_MALLOC_HEAP_SIZE, heap_limit);
```
改用CUDA Runtime API的隐式上下文：如果业务允许，切换到cudaRuntime API，利用其自动上下文管理机制，减少手动创建上下文的内存开销，或结合线程局部存储复用上下文。

内容的提问来源于stack exchange，提问作者Javier Usón