卷积层与线性层转换时遇Trying to backward through the graph a second time报错

我在学习卷积神经网络（CNN）时接触到conv2d.zero_grad()语句，好奇哪些对象可以调用该方法。于是在一段可正常运行的卷积层代码基础上，将二维卷积层替换为线性层，却反复报错：Trying to backward through the graph a second time。

可正常运行的卷积层代码 #

import torch
from torch import nn


def corr2d(X, K):
    h, w = K.shape
    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))
    for i in range(Y.shape[0]):
        for j in range(Y.shape[1]):
            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()
    return Y


X = torch.ones((6, 8))
X[:, 2:6] = 0
K = torch.tensor([[1.0, -1.0]])
Y = corr2d(X, K)
conv2d = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=(1, 2), bias=False)
X = X.reshape((1, 1, 6, 8))
Y = Y.reshape((1, 1, 6, 7))
lr = 3e-2

for i in range(10):
    Y_hat = conv2d(X)
    print(Y_hat)
    l = (Y_hat - Y) ** 2
    conv2d.zero_grad()
    l.sum().backward()
    conv2d.weight.data[:] -= lr * conv2d.weight.grad
    if (i + 1) % 2 == 0:
        print(f"epoch{i + 1},loss{l.sum():.3f}")

# Can operate normally

报错的线性层代码 #

import torch
from torch import nn
X = torch.rand(size = (2,5),requires_grad = True)
true_w = torch.tensor([1,2,3,4,5.]).reshape((X.shape[1],-1))
true_b = torch.zeros(X.shape[0]).reshape((-1,1))
Y = torch.matmul(X,true_w) + true_b
Y += torch.rand(size=(Y.shape))
linear1 = nn.Linear(5,1)
lr = 0.01
for i in range(10):
    Y_hat = linear1(X)
    l = (Y_hat-Y)**2
    print(l)
    linear1.zero_grad()
    l.sum().backward()
    linear1.weight.data[:] -= lr*linear1.weight.grad
    if(i + 1) % 2 == 0:
        print(f"epoch{i+1},loss{l.sum():.3f}")
# Trying to backward through the graph a second time

问题解析与解决 #

1. 报错原因 #

两段代码的核心区别在于输入和标签是否参与计算图的梯度传播：

• 卷积代码里的X和Y都是普通张量（默认requires_grad=False），仅作为固定数据使用，每次迭代的计算图都是独立的，反向传播后计算图会被释放，不会有重复反向传播的问题。
• 线性层代码中，X被设置了requires_grad=True，而且Y是通过X计算出来的，这导致Y也被绑定到计算图中。第一次反向传播后，计算图没有被释放，第二次迭代时复用了这个带有历史记录的计算图，PyTorch默认不允许对同一计算图多次反向传播，因此报错。

2. 关于zero_grad()的可调用对象 #

所有继承自nn.Module的模块（比如nn.Conv2d、nn.Linear，以及自定义的模型类）都能调用zero_grad()，它的作用是把该模块下所有可训练参数的梯度清零，避免梯度累加影响后续迭代。

3. 解决代码 #

只需要让X和Y不参与梯度计算即可，修改后的线性层代码如下：

import torch
from torch import nn
# 去掉X的requires_grad=True，默认就是False
X = torch.rand(size = (2,5))
true_w = torch.tensor([1,2,3,4,5.]).reshape((X.shape[1],-1))
true_b = torch.zeros(X.shape[0]).reshape((-1,1))
Y = torch.matmul(X,true_w) + true_b
Y += torch.rand(size=(Y.shape))
# 确保Y不参与梯度计算
Y.requires_grad_(False)
linear1 = nn.Linear(5,1)
lr = 0.01
for i in range(10):
    Y_hat = linear1(X)
    l = (Y_hat-Y)**2
    print(l)
    linear1.zero_grad()
    l.sum().backward()
    linear1.weight.data[:] -= lr*linear1.weight.grad
    if(i + 1) % 2 == 0:
        print(f"epoch{i+1},loss{l.sum():.3f}")

内容的提问来源于stack exchange，提问作者fightboy