使用Stable-Baselines3 PPO训练Super Mario Bros遇render_mode属性错误求助

环境信息 #

• Stable-Baselines3：2.0.0
• gym-super-mario-bros：7.4.0
• gym：0.23.1
• 运行环境：Colab

训练代码 #

from nes_py.wrappers import JoypadSpace
import gym_super_mario_bros
from gym_super_mario_bros.actions import SIMPLE_MOVEMENT
import time
from matplotlib import pyplot as plt
from stable_baselines3 import PPO
import gym

env = gym_super_mario_bros.make('SuperMarioBros-v0')
env = JoypadSpace(env, SIMPLE_MOVEMENT)

tensorboard_log = r'./tensorboard_log/'

model = PPO("CnnPolicy", env, verbose=1,tensorboard_log = tensorboard_log)
model.learn(total_timesteps=25000)
model.save("mario_model")

运行时错误 #

Using cpu device
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-73-f912901bbdee> in <cell line: 6>()
      4 
      5 #model = PPO("CnnPolicy", env, verbose=1,tensorboard_log = tensorboard_log)
----> 6 model = PPO('MlpPolicy', env, verbose=1)
      7 model.learn(total_timesteps=25000)
      8 model.save("mario_model")

5 frames
/usr/local/lib/python3.10/dist-packages/shimmy/openai_gym_compatibility.py in __init__(self, env_id, make_kwargs, env)
     88 
     89         self.metadata = getattr(self.gym_env, "metadata", {"render_modes": []})
---> 90         self.render_mode = self.gym_env.render_mode
     91         self.reward_range = getattr(self.gym_env, "reward_range", None)
     92         self.spec = getattr(self.gym_env, "spec", None)

AttributeError: 'SuperMarioBrosEnv' object has no attribute 'render_mode'

降级Stable-Baselines3时的错误 #

Collecting stable-baselines3==1.5.0
  Using cached stable_baselines3-1.5.0-py3-none-any.whl (177 kB)
Collecting gym==0.21 (from stable-baselines3==1.5.0)
  Using cached gym-0.21.0.tar.gz (1.5 MB)
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

解决方案 #

方案1：无需降级，修复环境兼容问题 #

错误根源是gym-super-mario-bros 7.4.0基于旧版gym接口，没有render_mode属性，但Stable-Baselines3 2.0.0通过Shimmy转换环境时需要该属性。只需在创建环境后手动添加该属性即可：

修改后的环境创建代码：

env = gym_super_mario_bros.make('SuperMarioBros-v0')
# 手动添加render_mode属性，Colab环境推荐用rgb_array
env.render_mode = "rgb_array"
env = JoypadSpace(env, SIMPLE_MOVEMENT)

如果仍有问题，可使用Shimmy的兼容包装器：

from shimmy.openai_gym_compatibility import OpenAIGymCompatibilityV0ToV3

env = gym_super_mario_bros.make('SuperMarioBros-v0')
env = JoypadSpace(env, SIMPLE_MOVEMENT)
env = OpenAIGymCompatibilityV0ToV3(env)

方案2：正确降级Stable-Baselines3及依赖 #

降级到SB3 1.5.0时，pip强制安装的gym 0.21.0不支持Python3.10（Colab当前默认版本），需指定兼容的gym版本：

执行以下安装命令：

pip uninstall -y gym
pip install stable-baselines3==1.5.0 gym==0.23.1 gym-super-mario-bros==7.4.0

若依赖仍有冲突，可添加旧版解析器参数：

pip install stable-baselines3==1.5.0 gym==0.23.1 gym-super-mario-bros==7.4.0 --use-deprecated=legacy-resolver

内容的提问来源于stack exchange，提问作者Catalyst