gym环境step方法报错ValueError：解包值数量不符求助

问题情况 #

运行CartPole-v0强化学习教程代码时，执行n_state, reward, done, info = env.step(action)行出现ValueError: too many values to unpack (expected 4)错误，教程视频中代码可正常运行，但本地执行始终报错。

执行代码 #

episodes = 10
for episode in range(1, episodes+1):
    state = env.reset()
    done = False
    score = 0 
    
    while not done:
        env.render()
        action = random.choice([0,1])
        n_state, reward, done, info = env.step(action)
        score+=reward
    print('Episode:{} Score:{}'.format(episode, score))

报错栈 #

ValueError                                Traceback (most recent call last)
Cell In[51], line 10
      8     env.render()
      9     action = random.choice([0,1])
---> 10     n_state, reward, done, info = env.step(action)
     11     score+=reward
     12 print('Episode:{} Score:{}'.format(episode, score))

ValueError: too many values to unpack (expected 4)

导入库及环境初始化代码 #

import os
import gym
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.evaluation import evaluate_policy

environment_name = "CartPole-v0"

env = gym.make(environment_name)

解决方案 #

这个错误是Gym版本差异导致的：

• 旧版Gym（0.25及以下）中，env.step()返回4个值：(observation, reward, done, info)
• 新版Gym（0.26及以上）中，API更新，将原done拆分为terminated（环境正常终止）和truncated（因步数限制等被截断），返回5个值：(observation, reward, terminated, truncated, info)

方法1：适配新版Gym #

直接修改解包逻辑，合并terminated和truncated为原逻辑中的done：

episodes = 10
for episode in range(1, episodes+1):
    state = env.reset()
    done = False
    score = 0 
    
    while not done:
        env.render()
        action = random.choice([0,1])
        # 修改解包行，合并terminated和truncated为done
        n_state, reward, terminated, truncated, info = env.step(action)
        done = terminated or truncated
        score+=reward
    print('Episode:{} Score:{}'.format(episode, score))

方法2：兼容新旧版本 #

如果需要代码同时适配新旧Gym版本，可以通过判断返回值长度处理：

episodes = 10
for episode in range(1, episodes+1):
    state = env.reset()
    done = False
    score = 0 
    
    while not done:
        env.render()
        action = random.choice([0,1])
        step_result = env.step(action)
        if len(step_result) == 5:
            n_state, reward, terminated, truncated, info = step_result
            done = terminated or truncated
        else:
            n_state, reward, done, info = step_result
        score+=reward
    print('Episode:{} Score:{}'.format(episode, score))

内容的提问来源于stack exchange，提问作者RFM