如何在Python运行中的脚本里关闭已启用的GPU？

Hey there! I totally get your frustration—trying to toggle GPU off mid-script by changing CUDA_VISIBLE_DEVICES to -1 doesn’t work, right? Let’s first break down why that approach fails, then jump into practical, working solutions.

Why Modifying CUDA_VISIBLE_DEVICES Mid-Script Doesn’t Work #

The CUDA_VISIBLE_DEVICES environment variable is only read by the CUDA runtime when your process first starts up. Once your script has already loaded CUDA libraries and initialized GPU resources, changing this variable later does absolutely nothing—CUDA doesn’t re-check it during runtime. So that’s why your attempt didn’t work.

Practical Solutions by Framework #

Below are targeted fixes for the two most common ML frameworks:

PyTorch

1. Move all models and tensors to CPU
   First, you need to explicitly transfer every piece of your model and data from GPU to CPU:

   # Move your trained model to CPU
   model = model.to("cpu")
   
   # If you have any active tensors stored on GPU, transfer those too
   # For example, if you have a test dataset on GPU:
   test_data = test_data.to("cpu")
   

2. Clear CUDA cache to free GPU memory
   Even after moving things to CPU, some residual GPU memory might be held. Clear it with:

   import torch
   torch.cuda.empty_cache()
   

Now any subsequent evaluation code will run purely on CPU, and your GPU will be released for other tasks.

TensorFlow

1. Reset the session and switch to CPU device
   TensorFlow ties computations to devices explicitly, so wrap your evaluation code in a CPU device context:

   import tensorflow as tf
   
   # Clear existing GPU sessions to free resources
   tf.keras.backend.clear_session()
   
   # Run your evaluation code on CPU
   with tf.device("/CPU:0"):
       # Example evaluation code
       test_loss, test_acc = model.evaluate(test_dataset)
       print(f"Test accuracy: {test_acc}")
   

2. Optional: Disable GPU access entirely mid-script
   If you want to ensure no future code uses GPU, you can unconfigure GPU devices:

   gpus = tf.config.list_physical_devices("GPU")
   if gpus:
       # Disable access to all GPUs
       tf.config.set_visible_devices([], "GPU")
   

General Tips #

• No matter which framework you use, the core idea is explicitly migrating all computations to CPU and releasing GPU resources. Environment variable tweaks won’t cut it once the script is running.
• Double-check that all parts of your evaluation pipeline (models, data, helper functions) are using CPU—sometimes hidden tensors might still be on GPU if you miss them.

内容的提问来源于stack exchange，提问作者Jürgen K.