Gymnasium自定义环境报'too many values to unpack'错误求助

环境定义（动作/观测空间） #

动作空间范围：

min_action = np.array([-5, -5] * len(self.agents), dtype=np.float32)
max_action = np.array([5, 5] * len(self.agents), dtype=np.float32)

观测空间范围：

min_obs = np.array([-np.inf, -np.inf, -2.5, -2.5] * len(self.agents), dtype=np.float32)
max_obs = np.array([np.inf, np.inf, 2.5, 2.5] * len(self.agents), dtype=np.float32)

训练代码 #

import numpy as np
import torch as th
from Parameters import *
from stable_baselines3 import PPO
from main import FlockingEnv, CustomMultiAgentPolicy
from Callbacks import TQDMProgressCallback, LossCallback
import os
from stable_baselines3.common.vec_env import DummyVecEnv


if os.path.exists(Results["Rewards"]):
    os.remove(Results["Rewards"])
    print(f"File {Results['Rewards']} has been deleted.")

if os.path.exists("training_rewards.json"):
    os.remove("training_rewards.json")
    print(f"File training_rewards has been deleted.")    

def seed_everything(seed):
    np.random.seed(seed)
    os.environ['PYTHONHASHSEED'] = str(seed)
    th.manual_seed(seed)
    th.cuda.manual_seed(seed)
    th.backends.cudnn.deterministic = True
    env.seed(seed)
    env.action_space.seed(seed)


loss_callback = LossCallback()
env = DummyVecEnv([lambda: FlockingEnv()])

seed_everything(SimulationVariables["Seed"])

# Model Training
model = PPO(CustomMultiAgentPolicy, env, tensorboard_log="./ppo_Agents_tensorboard/", verbose=1)
model.set_random_seed(SimulationVariables["ModelSeed"])
progress_callback = TQDMProgressCallback(total_timesteps=SimulationVariables["LearningTimeSteps"])
# Train the model
model.learn(total_timesteps=SimulationVariables["LearningTimeSteps"], callback=[progress_callback, loss_callback])

报错信息 #

Using cuda device
Traceback (most recent call last):
File "D:\Thesis_\FlockingFinal\MultiAgentFlocking\Training.py", line 45, in <module>
model.learn(total_timesteps=SimulationVariables["LearningTimeSteps"], callback=[progress_callback, loss_callback])      
File "C:\Python312\Lib\site-packages\stable_baselines3\ppo\ppo.py", line 315, in learn
return super().learn(
            ^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\stable_baselines3\common\on_policy_algorithm.py", line 287, in learn
total_timesteps, callback = self._setup_learn(
                                 ^^^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\stable_baselines3\common\base_class.py", line 423, in _setup_learn
self._last_obs = self.env.reset()  # type: ignore[assignment]
                      ^^^^^^^^^^^^^^^^
File "C:\Python312\Lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 77, in reset
obs, self.reset_infos[env_idx] = self.envs[env_idx].reset(seed=self._seeds[env_idx], **maybe_options)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)

问题原因与解决方法 #

• 核心问题是自定义FlockingEnv的reset()方法返回值不符合Gymnasium/Stable Baselines3的API规范。
• Gymnasium要求reset()必须返回(observation, info)二元组；如果你的环境返回了更多值（比如旧Gym格式的obs, reward, done, info），或者只返回了单个观测值，都会导致DummyVecEnv解包时抛出这个错误。
• 修复步骤：
  1. 打开FlockingEnv的定义文件，找到reset()方法
  2. 如果方法只返回观测值，修改为返回(observation, {})（info参数传空字典即可）
  3. 如果方法返回了多余的参数（比如旧版Gym的四元组），调整为仅返回观测值和info字典的二元组
• 额外注意：Stable Baselines3对Gymnasium环境的step()方法也有要求，必须返回(observation, reward, terminated, truncated, info)五元组，确保你的环境同时满足这个规范。

内容的提问来源于stack exchange，提问作者Hamza