在CartPole-v1环境创建Stable-Baselines3 PPO模型时出现段错误

环境配置 #

• Python版本：3.9、3.10（均使用Conda环境）
• 依赖安装方式：
  • 通过pip安装stable-baselines[extra]
  • 通过conda安装OpenAI Gym库

错误表现 #

• Python 3.10环境下，错误触发于threading.py文件的wait函数调用位置
• Python 3.9环境下，错误触发于constraints.py文件的_IntegerInterval.check方法中
• 注释掉代码中的model.learn()和model.save()后，错误不再出现，说明问题与模型训练流程直接相关

示例代码 #

import sys
import gym
from stable_baselines3 import PPO, A2C
from stable_baselines3.common.env_util import make_vec_env

import faulthandler

faulthandler.enable()

def main(): 
  print("Going to create model")
  env = gym.make("CartPole-v1")

  model = PPO("MlpPolicy", env, verbose=1)
  print("Model created")
  model.learn(total_timesteps=25000)
  model.save("ppo_cartpole")

if __name__ == '__main__':
    main()
    sys.settrace(None)

终端输出（Python 3.9环境） #

(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$  cd /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC ; /usr/bin/env /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python /Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 50143 -- /Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py 
Going to create model
Backend MacOSX is interactive backend. Turning interactive mode on.
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Fatal Python error: Segmentation fault

Thread 0x0000700007a7b000 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 316 in wait
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/threading.py", line 581 in wait
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 261 in _on_run
  File "/Users/ilijavuk/.vscode/extensions/ms-python.python-2022.20.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_daemon_thread.py", line 49 in run
(py39) ilijas-mbp:Doom_DQN_GC ilijavuk$ 

GDB调试补充信息 #

运行时调试输出 #

Starting program: /Users/ilijavuk/opt/anaconda3/envs/py39/bin/python StableBaselinesTest.py
[New Thread 0x1603 of process 2803]
[New Thread 0x2003 of process 2803]
warning: unhandled dyld version (17)
Going to create model
Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.
Model created
[New Thread 0x1807 of process 2803]
[New Thread 0x2103 of process 2803]
[New Thread 0x2203 of process 2803]
[New Thread 0x2303 of process 2803]
[New Thread 0x2403 of process 2803]
[New Thread 0x2503 of process 2803]
[New Thread 0x2603 of process 2803]
[New Thread 0x2703 of process 2803]
[New Thread 0x2803 of process 2803]
[New Thread 0x2903 of process 2803]
[New Thread 0x2a03 of process 2803]
[New Thread 0x2b03 of process 2803]
[New Thread 0x3e03 of process 2803]
[New Thread 0x3f03 of process 2803]

Thread 3 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x1807 of process 2803]
0x0000000000000000 in ?? ()
(gdb) backtrace
#0  0x0000000000000000 in ?? ()
#1  0x0000000103a48538 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb)

退出GDB后的错误回溯 #

(gdb) q
A debugging session is active

        Inferior 1 [process 2803] will be killed.

Quit anyway? (y or n) y
Fatal Python error: Segmentation fault

Thread 0x00007ff847bb84c0 (most recent call first):
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 114 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/container.py", line 204 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/torch_layers.py", line 263 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/policies.py", line 627 in forward
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194 in _call_impl
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 166 in collect_rollouts
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 248 in learn
  File "/Users/ilijavuk/opt/anaconda3/envs/py39/lib/python3.9/site-packages/stable_baselines3/ppo/ppo.py", line 307 in learn
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 16 in main
  File "/Users/ilijavuk/Documents/Reinforcement_Learning/Doom_DQN_GC/StableBaselinesTest.py", line 20 in <module>

内容的提问来源于stack exchange，提问作者Ilija Vuk