Gymnasium渲染强制开启问题：Q-Learning代码无法关闭渲染

我是Gym新手，编写了一个简单的Q-Learning程序，但遇到奇怪问题：无法关闭渲染，导致程序运行耗时极长。

我的代码 #

import gymnasium as gym
import numpy as np

env = gym.make("MountainCar-v0", render_mode="human")

LEARNING_RATE = 0.1
DISCOUNT = 0.95
EPISODES = 25000

SHOW_EVERY = 500

DISCRETE_OS_SIZE = [20] * len(env.observation_space.low)
discrete_os_win_size = (env.observation_space.high - env.observation_space.low) / DISCRETE_OS_SIZE

q_table = np.random.uniform(low=-2, high=0, size=(DISCRETE_OS_SIZE + [env.action_space.n]))


def get_discrete_state(state):
    discrete_state = (state - env.observation_space.low) / discrete_os_win_size
    return tuple(discrete_state.astype(int))


for episode in range(EPISODES):
    if episode % SHOW_EVERY == 0:
        render = True
    else:
        render = False
    
    print("Episode:", episode)
    
    discrete_state = get_discrete_state(tuple(env.reset()[0].astype(int)))
    
    done = False
    while not done:
        action = np.argmax(q_table[discrete_state])
        new_state, reward, terminated, truncated, _ = env.step(action)
        done = truncated or terminated
        
        new_discrete_state = get_discrete_state(new_state)
        
        # Rendering the episode
        # (Even removing this part does not help)
        if render:
            env.render()

        if not done:
            # Updating the Q-table
            max_future_q = np.max(q_table[new_discrete_state])
            current_q = q_table[discrete_state + (action, )]
            new_q = (1-LEARNING_RATE)* current_q + LEARNING_RATE * (reward + DISCOUNT * max_future_q)
            
            q_table[discrete_state + (action, )] = new_q
       
        # If the car made it to the goal
        elif new_state[0] >= env.unwrapped.goal_position:
            q_table[discrete_state + (action, )] = 0
            print("MADE IT ON EPISODE:", episode)
        discrete_state = new_discrete_state
    
env.close()

已尝试的方法 #

• 移除env.render()代码：无效
• 手动替换初始discrete_state为默认值(13,10)：虽能关闭渲染，但指定轮次也无法渲染

解决方案 #

核心问题 #

你初始化环境时直接指定了render_mode="human"，这个参数会强制创建一个可视化窗口，哪怕你不调用env.render()，窗口也会一直后台运行并占用大量资源，这就是移除env.render()也没用的原因。

修复步骤 #

1. 动态切换渲染模式：初始化环境时不指定render_mode（默认无渲染），仅在需要展示的轮次（如每500次）重新创建带render_mode="human"的环境，展示结束后切回无渲染环境。
2. 修正状态离散化的错误：代码中get_discrete_state(tuple(env.reset()[0].astype(int)))里的.astype(int)是多余的，会导致状态值被错误截断，直接用env.reset()[0]作为输入即可。

修改后的完整代码 #

import gymnasium as gym
import numpy as np

# 初始化无渲染模式的环境
env = gym.make("MountainCar-v0")

LEARNING_RATE = 0.1
DISCOUNT = 0.95
EPISODES = 25000
SHOW_EVERY = 500

DISCRETE_OS_SIZE = [20] * len(env.observation_space.low)
discrete_os_win_size = (env.observation_space.high - env.observation_space.low) / DISCRETE_OS_SIZE
q_table = np.random.uniform(low=-2, high=0, size=(DISCRETE_OS_SIZE + [env.action_space.n]))


def get_discrete_state(state):
    discrete_state = (state - env.observation_space.low) / discrete_os_win_size
    return tuple(discrete_state.astype(int))


for episode in range(EPISODES):
    render = episode % SHOW_EVERY == 0
    
    # 根据需要切换环境的渲染模式
    if render:
        # 关闭之前的无渲染环境，创建带可视化的环境
        env.close()
        env = gym.make("MountainCar-v0", render_mode="human")
    elif episode % SHOW_EVERY == 1:
        # 渲染结束后切回无渲染环境，节省资源
        env.close()
        env = gym.make("MountainCar-v0")
    
    print("Episode:", episode)
    # 修正：去掉不必要的astype(int)
    discrete_state = get_discrete_state(env.reset()[0])
    
    done = False
    while not done:
        action = np.argmax(q_table[discrete_state])
        new_state, reward, terminated, truncated, _ = env.step(action)
        done = truncated or terminated
        
        new_discrete_state = get_discrete_state(new_state)
        
        if render:
            env.render()

        if not done:
            max_future_q = np.max(q_table[new_discrete_state])
            current_q = q_table[discrete_state + (action, )]
            new_q = (1-LEARNING_RATE)* current_q + LEARNING_RATE * (reward + DISCOUNT * max_future_q)
            q_table[discrete_state + (action, )] = new_q
        elif new_state[0] >= env.unwrapped.goal_position:
            q_table[discrete_state + (action, )] = 0
            print("MADE IT ON EPISODE:", episode)
        
        discrete_state = new_discrete_state
    
env.close()

额外提示 #

• 每次切换环境时务必调用env.close()，避免残留的窗口进程占用系统资源。
• 如果使用Gymnasium 0.26及以上版本，也可以直接调用env.set_render_mode("human")和env.set_render_mode(None)来切换，无需重新创建环境，会更高效。

内容的提问来源于stack exchange，提问作者Bitron