模型全量权重未更新且梯度为None的问题排查求助

我正在构建一个成绩预测模型，该模型使用经数学建模计算得到的值，通过RNN进行运算。但从数学建模所用权重到RNN的所有权重均未更新，经检查发现各权重的梯度均输出为None。

模型代码 #

class Expector(nn.Module):
  def __init__(self, input_size, hidden_size, num_layers):
    super(Expector, self).__init__()

    self.weight_0 = nn.Parameter(torch.tensor([0.2]))
    self.weight_1 = nn.Parameter(torch.tensor([0.8, 0.7, 0.6, 0.5])) #homeownership weight
    self.weight_2 = nn.Parameter(torch.tensor([0.5 for i in range(0, 13)])) #loan_purpose weight

    #interesst_rate_expector
    self.num_features = 13
    self.linear_1 = torch.nn.Linear(self.num_features, self.num_features*2)
    self.linear_2 = torch.nn.Linear(self.num_features*2, self.num_features*4)
    self.linear_3 = torch.nn.Linear(self.num_features*4, self.num_features*8)
    self.linear_out = torch.nn.Linear(self.num_features*8, 1)

    #loan_rating_expector
    self.rnn = nn.RNN(input_size=input_size, hidden_size=hidden_size, num_layers=num_layers, batch_first=True)
    self.fc = nn.Linear(hidden_size, input_size)

  def interest_rate_expector(self, x):
    x = [x[i] for i in range(len(x)-1)]
    for i in [0,4,9,10,11]:
      if x[i] > 0:
        x[i] = math.log(x[i])
    input = torch.tensor(x).to(DEVICE)
    out1 = self.linear_1(input)
    out1 = torch.nn.functional.softplus(out1)
    out2 = self.linear_2(out1)
    out2 = torch.nn.functional.softplus(out2)
    out3 = self.linear_3(out2)
    out3 = torch.nn.functional.softplus(out3)
    logits = self.linear_out(out3)
    interest_rate = torch.sigmoid(logits)

    return interest_rate

  def mathematical_modeling(self, x):
    salary = x[4] / 12 if x[4] != 0 else 300000
    debt = x[4] * (1 / x[5]) if x[5] != 0 else 0
  
    interest_rate = float(self.interest_rate_expector(x))

    if x[2] > 4:
        repayment = (x[0] * interest_rate * ((1 + interest_rate) ** x[1])) / ((1 + interest_rate) ** x[1]) - 1
        result = (debt * self.weight_0[0] + repayment) / (salary * x[3])
    else:
        result = (debt * self.weight_0[0] + x[0] * x[1]) / (salary * x[3])

    return torch.tensor(result)

  def forward(self, x):
    x[3] = torch.matmul(one_hot_encoding1(x[3]), self.weight_1.view(-1, 1))
    x[7] = torch.matmul(one_hot_encoding2(x[7]), self.weight_2.view(-1, 1))

    tensor = torch.tensor([x[6], x[7], x[8], x[10], x[11], x[12], self.mathematical_modeling(x)])

    tensor = tensor.unsqueeze(0)
    output, _ = self.rnn(tensor.to(DEVICE))
    final_output = self.fc(output)
    output_probabilities = F.softmax(final_output, dim=1)
    
    return output_probabilities

已尝试的操作 #

• 显式设置requires_grad=True；
• 让mathematical_modeling函数返回tensor；
• 调整学习率。

问题根源分析 #

梯度为None的核心原因是计算图被手动操作切断，具体有以下几个关键点：

1. 将tensor转为Python数值破坏梯度链：mathematical_modeling中interest_rate = float(self.interest_rate_expector(x))把带梯度的tensor转成了Python float，直接切断了interest_rate_expector中线性层权重到后续计算的梯度传播。
2. 列表操作与手动创建tensor丢失梯度：interest_rate_expector中把tensor转成列表x = [x[i] for i in range(len(x)-1)]，再用torch.tensor(x)重新创建，会丢失原tensor的梯度信息；同理forward中用torch.tensor()拼接元素时，也会丢失输入tensor的梯度。
3. 使用Python数学函数而非PyTorch张量函数：interest_rate_expector中用math.log()处理数值，这是对Python标量的操作，无法保留梯度，应该用torch.log()处理tensor。
4. 条件分支中的数值计算未保留tensor属性：mathematical_modeling中的条件分支里，用Python数值计算的部分没有维持tensor类型，导致梯度无法传递到weight_0等参数。

修复后的代码 #

class Expector(nn.Module):
  def __init__(self, input_size, hidden_size, num_layers):
    super(Expector, self).__init__()

    self.weight_0 = nn.Parameter(torch.tensor([0.2]))
    self.weight_1 = nn.Parameter(torch.tensor([0.8, 0.7, 0.6, 0.5])) #homeownership weight
    self.weight_2 = nn.Parameter(torch.tensor([0.5 for i in range(0, 13)])) #loan_purpose weight

    #interesst_rate_expector
    self.num_features = 13
    self.linear_1 = torch.nn.Linear(self.num_features, self.num_features*2)
    self.linear_2 = torch.nn.Linear(self.num_features*2, self.num_features*4)
    self.linear_3 = torch.nn.Linear(self.num_features*4, self.num_features*8)
    self.linear_out = torch.nn.Linear(self.num_features*8, 1)

    #loan_rating_expector
    self.rnn = nn.RNN(input_size=input_size, hidden_size=hidden_size, num_layers=num_layers, batch_first=True)
    self.fc = nn.Linear(hidden_size, input_size)

  def interest_rate_expector(self, x):
    # 用tensor索引替代列表操作，保留梯度
    x = x[:-1]
    # 对tensor使用torch.log，保留梯度
    mask = x[[0,4,9,10,11]] > 0
    x[[0,4,9,10,11]] = torch.where(mask, torch.log(x[[0,4,9,10,11]]), x[[0,4,9,10,11]])
    input = x.to(DEVICE)
    out1 = self.linear_1(input)
    out1 = torch.nn.functional.softplus(out1)
    out2 = self.linear_2(out1)
    out2 = torch.nn.functional.softplus(out2)
    out3 = self.linear_3(out2)
    out3 = torch.nn.functional.softplus(out3)
    logits = self.linear_out(out3)
    interest_rate = torch.sigmoid(logits)

    return interest_rate

  def mathematical_modeling(self, x):
    # 用tensor操作替代Python条件判断，保留梯度
    salary = torch.where(x[4] != 0, x[4] / 12, torch.tensor(300000.0, device=DEVICE))
    debt = torch.where(x[5] != 0, x[4] * (1 / x[5]), torch.tensor(0.0, device=DEVICE))
  
    # 保留tensor类型，不转float
    interest_rate = self.interest_rate_expector(x)

    # 用torch.where处理条件分支，维持计算图
    repayment_case1 = (x[0] * interest_rate * ((1 + interest_rate) ** x[1])) / ((1 + interest_rate) ** x[1]) - 1
    result_case1 = (debt * self.weight_0[0] + repayment_case1) / (salary * x[3])
    result_case2 = (debt * self.weight_0[0] + x[0] * x[1]) / (salary * x[3])
    result = torch.where(x[2] > 4, result_case1, result_case2)

    return result

  def forward(self, x):
    x[3] = torch.matmul(one_hot_encoding1(x[3]), self.weight_1.view(-1, 1))
    x[7] = torch.matmul(one_hot_encoding2(x[7]), self.weight_2.view(-1, 1))

    # 用torch.stack拼接tensor，保留梯度
    tensor = torch.stack([x[6], x[7], x[8], x[10], x[11], x[12], self.mathematical_modeling(x)])

    tensor = tensor.unsqueeze(0)
    output, _ = self.rnn(tensor.to(DEVICE))
    final_output = self.fc(output)
    output_probabilities = F.softmax(final_output, dim=1)
    
    return output_probabilities

修复说明 #

• 所有涉及tensor的操作都保留tensor类型，避免转为Python数值；
• 用torch.where替代Python的if-else和条件赋值，维持计算图的连续性；
• 用tensor索引和torch.stack替代列表操作与手动创建tensor，确保梯度不丢失；
• 替换math.log为torch.log，保证梯度能正常传播。

内容的提问来源于stack exchange，提问作者young