PyTorch优化问题代码无法调用GPU，仅用CPU运行求助

Hey Nicolai, let's troubleshoot why your PyTorch code isn't leveraging your GPU even though your setup supports it. I'll walk you through the most common issues and fixes step by step:

1. First, Confirm PyTorch Can See Your GPU #

Before diving into code changes, verify that PyTorch is correctly detecting your CUDA-enabled GPU. Run this quick check:

import torch
print("CUDA available:", torch.cuda.is_available())
print("Number of GPUs:", torch.cuda.device_count())
if torch.cuda.is_available():
    print("GPU name:", torch.cuda.get_device_name(0))

• If torch.cuda.is_available() returns False, your PyTorch installation might be the CPU-only version, or your CUDA driver is outdated/misconfigured. Reinstall PyTorch with the correct CUDA version (match your system's CUDA version to the PyTorch build).
• If it returns True, move on to the next steps.

2. Move Tensors, Models, and Variables to the GPU #

This is the #1 mistake most people make: forgetting to transfer all computation-related data to the GPU. Every tensor, model parameter, and Variable involved in your optimization needs to live on the GPU.

Example Fixes: #

• For tensors: Use .to(device) (recommended) or .cuda() (older syntax) to move them. First define a device variable for portability:

  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
  
  # Convert numpy arrays to PyTorch tensors and move to GPU
  np_array = np.random.rand(100)
  torch_tensor = torch.from_numpy(np_array).float().to(device)
  
  # Create tensors directly on GPU
  random_tensor = torch.randn(100, device=device)
  

• For Variable instances (note: in newer PyTorch versions, Variable is deprecated—you can just use tensors directly with requires_grad=True):

  # Old way with Variable
  x = Variable(torch.randn(n)).to(device)
  # Newer way (preferred)
  x = torch.randn(n, requires_grad=True, device=device)
  

• For models (if you're using a custom model or pre-trained one):

  model = YourCustomModel()
  model = model.to(device)
  

3. Ensure Optimizers Are Initialized After Moving Models to GPU #

If you initialize your optimizer before moving the model to the GPU, the optimizer will be tracking CPU-based parameters instead of GPU ones. Always do this in the right order:

# ❌ Wrong: Optimizer is initialized before moving model to GPU
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
model = model.to(device)

# ✅ Correct: Move model to GPU first, then initialize optimizer
model = model.to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

4. Avoid Mixing CPU and GPU Operations #

Your code uses numpy and scipy functions (like cmdscale with numpy arrays). When you convert these results back to PyTorch tensors, make sure to move them to the GPU immediately. Also, note that matplotlib requires CPU data for plotting, so you'll need to move tensors back to CPU before plotting:

# After running cmdscale (numpy output)
result_np = cmdscale(D)
result_tensor = torch.from_numpy(result_np).to(device)

# When plotting with matplotlib
plt.plot(result_tensor.cpu().numpy())

Mixing GPU tensors with numpy/scipy operations will force PyTorch to move data back to CPU, which kills GPU utilization.

5. Verify Computation Is Running on GPU #

To confirm your code is actually using the GPU:

• Check the device of a tensor: print(tensor_data.device) (should show cuda:0 or similar)
• Monitor GPU usage in real-time: Use nvidia-smi in the terminal (Linux/macOS) or the Task Manager's GPU tab (Windows) while your code runs—you should see a spike in usage if the GPU is working.

Final Quick Checklist #

• PyTorch detects your GPU (torch.cuda.is_available() == True)
• All tensors, model parameters, and trainable variables are on the GPU
• Optimizer is initialized after moving the model to GPU
• No unintended CPU-GPU data transfers mid-computation

内容的提问来源于stack exchange，提问作者NicolaiF