PyTorch报错：matmul参数需至少1D，现为0D与2D（强化学习场景）

我正在学习PyTorch及基于该库的强化学习，对batch size概念不太理解，运行代码时持续报错：

RuntimeError: both arguments to matmul need to be at least 1D, but they are 0D and 2D

以下是我的代码：

import numpy as np
import gym
import torch as T
import torch.multiprocessing as mp
import torch.nn as nn
import torch.nn.functional as F
from torch.distributions import Categorical
import torch.optim as optim


class network(nn.Module):
    def __init__(self, lr, input_dims, h1_dims, h2_dims, n_actions):
        super(network, self).__init__()

        self.lr = lr
        self.input_dims = input_dims
        self.h1_dims = h1_dims
        self.h2_dims = h2_dims
        self.n_actions = n_actions

        self.h1 = nn.Linear(self.input_dims, self.h1_dims)
        self.h2 = nn.Linear(self.h1_dims, self.h2_dims)
        self.p = nn.Linear(self.h2_dims, self.n_actions)
        self.v = nn.Linear(self.h2_dims, 1)

        self.optimizer = optim.Adam(self.parameters(), lr=self.lr)
        self.device = T.device('cuda:0' if T.cuda.is_available() else 'cpu')
        self.to(self.device)

    def forward(self, observation):
        state = T.tensor(observation, dtype=T.float).to(self.device)

        x = F.relu(self.h1(state))
        x = F.relu(self.h2(x))
        v = self.v(x)
        p = F.softmax(self.p(x), dim=0)

        return v, p



class agent(object):

      def __init__(self, n_actions):

            self.gamma = 0.99
            self.n_actions = n_actions
            self.log_probs = None
            self.network = network(lr=0.005, input_dims=4, h1_dims=128,
                  h2_dims=128, n_actions=self.n_actions)


      def choose_action(self, observation):

              value, policy = self.network(observation)

              action_probs = T.distributions.Categorical(probs=policy)
              action = action_probs.sample()
              log_probs = action_probs.log_prob(action)
              self.log_probs = log_probs
              return action.numpy()


      def learn(self, state, reward, new_state, done):
              self.network.optimizer.zero_grad()

              curr_state = np.array(state)
              next_state = np.array(new_state)
              value_current = self.network.forward(T.from_numpy(curr_state))
              value_next = self.network.forward(T.from_numpy(next_state))
              reward = T.tensor(reward, dtype=T.float).to(self.network.device)
              delta = reward + self.gamma*value_next[0]*(1-int(done)) - value_current[0]

              actor_loss = -self.log_probs * delta
              critic_loss = delta**2

              (actor_loss + critic_loss).backward()
              self.network.optimizer.step()




env = gym.make('CartPole-v1')

action_spaces = env.action_space.n

create_agent = agent(action_spaces)
score_history = []
num_episodes = 1

for i in range(num_episodes):
    done = False
    score = 0
    observation = env.reset()
    while not done:
        action = create_agent.choose_action(observation[0])
        observation_, reward, done, truncated, info = env.step(action)
        create_agent.learn(observation[0], reward, observation_, done)
        observation = observation_
        score += reward

        #print(learn)
    score_history.append(score)
    print('episode ', i, 'score %.2f' % score )

报错信息：

Exception has occurred: RuntimeError both arguments to matmul need to
be at least 1D, but they are 0D and 2D   File
"D:\kuliah\RL\policy_gradient2\testing.py", line 33, in forward
x = F.relu(self.h1(state))   File "D:\kuliah\RL\policy_gradient2\testing.py", line 55, in choose_action
value, policy = self.network(observation)   File "D:\kuliah\RL\policy_gradient2\testing.py", line 96, in 
action = create_agent.choose_action(observation[0]) RuntimeError: both arguments to matmul need to be at least 1D, but they are 0D and
2D

一、报错修复 #

1. 核心错误原因 #

报错源于两个问题：

• 主循环中错误传入了状态的单个元素（标量，0D），但网络定义的输入维度是4（对应CartPole的完整4维状态）；
• nn.Linear层要求输入至少为1维向量，标量输入会触发矩阵乘法维度不匹配的错误。

2. 具体修复步骤 #

（1）修正主循环传参

将主循环中传入单个状态元素的代码，改为传入完整状态：

# 原代码
action = create_agent.choose_action(observation[0])
create_agent.learn(observation[0], reward, observation_, done)

# 修改后
action = create_agent.choose_action(observation)
create_agent.learn(observation, reward, observation_, done)

（2）确保网络输入维度合法

修改network类的forward方法，强制将输入转为1维或带batch维度的张量，并调整softmax的维度：

def forward(self, observation):
    state = T.tensor(observation, dtype=T.float).to(self.device)
    # 确保输入至少为1维，若为标量则扩展为向量
    if state.dim() == 0:
        state = state.unsqueeze(0)
    # 增加batch维度（符合PyTorch常规输入格式）
    if state.dim() == 1:
        state = state.unsqueeze(0)
    x = F.relu(self.h1(state))
    x = F.relu(self.h2(x))
    v = self.v(x)
    # 针对batch维度做softmax
    p = F.softmax(self.p(x), dim=1)
    return v, p

（3）适配choose_action的输出维度

因为网络现在返回带batch维度的结果，需要调整动作采样逻辑：

def choose_action(self, observation):
    value, policy = self.network(observation)
    # 去除batch维度，适配单样本采样
    action_probs = T.distributions.Categorical(probs=policy.squeeze(0))
    action = action_probs.sample()
    log_probs = action_probs.log_prob(action)
    self.log_probs = log_probs
    return action.numpy()

（4）简化learn方法的输入处理

移除手动转张量的冗余代码，同时处理值函数的维度：

def learn(self, state, reward, new_state, done):
    self.network.optimizer.zero_grad()

    value_current = self.network(state)
    value_next = self.network(new_state)
    reward = T.tensor(reward, dtype=T.float).to(self.network.device)
    # 去除值函数的冗余维度，转为标量计算delta
    delta = reward + self.gamma*value_next[0].squeeze()*(1-int(done)) - value_current[0].squeeze()

    actor_loss = -self.log_probs * delta
    critic_loss = delta**2

    (actor_loss + critic_loss).backward()
    self.network.optimizer.step()

二、Batch Size概念解析 #

Batch Size指一次送入神经网络训练的样本数量，核心作用和特点：

• 单样本训练（Batch Size=1）：每次仅输入一个样本，训练波动大，但内存占用极低，适合你的当前单步更新强化学习场景；
• 批量训练（如Batch Size=32/64）：一次输入多个样本，损失为样本均值，训练更稳定，且能利用GPU并行计算加速；
• 在强化学习中，批量训练通常是收集多轨迹/多步的经验数据，攒成一个batch后再更新网络，比如PPO算法的经验回放机制，能有效提升训练效率和稳定性。

内容的提问来源于stack exchange，提问作者ryan chandra