PyTorch：最小化对称矩阵函数时出现反向传播错误

问题现象 #

运行代码时触发如下错误：

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

可复现代码：

import torch

A_nan = torch.tensor([[1.0, 2.0, torch.nan], [2.0, torch.nan, 5.0], [torch.nan, 5.0, 6.0]])
nan_idxs = torch.where(torch.isnan(torch.triu(A_nan)))
A_est = torch.clone(A_nan)
weights = torch.nn.ParameterList([])
for i, j in zip(*nan_idxs):
    w = torch.nn.Parameter(torch.distributions.Normal(3, 0.5).sample())
    A_est[i, j] = w
    A_est[j, i] = w
    weights.append(w)

optimizer = torch.optim.Adam(weights, lr=0.01)
for _ in range(10):
    optimizer.zero_grad()
    loss = torch.sum(A_est ** 2)
    loss.backward()
    optimizer.step()

问题原因 #

核心问题是A_est的生命周期与计算图绑定逻辑错误：

• 在训练循环外，直接将torch.nn.Parameter赋值给A_est的元素，导致A_est与这些参数的计算图永久绑定
• 第一次调用loss.backward()后，PyTorch会自动释放计算图的中间张量以节省内存
• 后续迭代仍使用同一个A_est计算loss，触发对已释放计算图的二次反向传播，引发错误

解决方案 #

将构建A_est的逻辑移到训练循环内部，每次迭代都基于当前参数值重新生成A_est，确保每次loss都关联全新的计算图。

修改后的代码：

import torch

A_nan = torch.tensor([[1.0, 2.0, torch.nan], [2.0, torch.nan, 5.0], [torch.nan, 5.0, 6.0]])
nan_idxs = torch.where(torch.isnan(torch.triu(A_nan)))
weights = torch.nn.ParameterList([])
# 仅初始化待训练参数，不提前绑定到A_est
for i, j in zip(*nan_idxs):
    w = torch.nn.Parameter(torch.distributions.Normal(3, 0.5).sample())
    weights.append(w)

optimizer = torch.optim.Adam(weights, lr=0.01)
for _ in range(10):
    optimizer.zero_grad()
    # 每次迭代重新构建A_est
    A_est = torch.clone(A_nan)
    for idx, (i, j) in enumerate(zip(*nan_idxs)):
        current_w = weights[idx]
        A_est[i, j] = current_w
        A_est[j, i] = current_w
    loss = torch.sum(A_est ** 2)
    loss.backward()
    optimizer.step()

关键改动说明 #

• 循环外仅初始化参数列表，避免提前将参数与A_est绑定
• 每次迭代从原始A_nan克隆新的A_est，并用当前参数值填充对称位置
• 确保每次训练的计算图独立，反向传播后可正常释放，不会出现重复使用已释放图的问题

内容的提问来源于stack exchange，提问作者Tendero