导入Keras Sequential遇unhashable type: 'list'错误，DQN模型运行失败

问题代码 #

from keras.models import Sequential
from keras.layers import Dense, Flatten
from keras.optimizers import Adam
from rl.agents import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory

class DeepQL:
    def __init__(self, env):
        self.env = env
        self.actions = env.action_space.n
        self.states  = env.observation_space.shape[0]

    def build_model(self, states, actions):
        model = Sequential()
        model.add(Flatten(input_shape=(1,states)))
        model.add(Dense(24, activation='relu'))
        model.add(Dense(24, activation='relu'))
        model.add(Dense(actions, activation='linear'))
        return model

    def build_agent(self):
        model = self.build_model(self.states, self.actions)
        policy = BoltzmannQPolicy()
        memory = SequentialMemory(limit=50000, window_length=1)
        dqn = DQNAgent(model=model, memory=memory, policy=policy, nb_actions=self.actions, nb_steps_warmup=10, target_model_update=1e-2)
        return dqn

    def compile(self):
        dqn = self.build_agent()
        dqn.compile(Adam(lr=1e-3), metrics=['mae'])
        dqn.fit(self.env, nb_steps=5000, visualize=False, verbose=1)

错误信息 #

Traceback (most recent call last):
  File "/Users/albi/PycharmProjects/replaybg_reinforce/py_replay_bg/reinforcementLearning/Agents/deepQL.py", line 1, in <module>
    from keras.models import Sequential
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/__init__.py", line 8, in <module>
    from keras import _tf_keras
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/_tf_keras/__init__.py", line 1, in <module>
    from keras._tf_keras import keras
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/_tf_keras/keras/__init__.py", line 8, in <module>
    from keras import activations
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/activations/__init__.py", line 8, in <module>
    from keras.src.activations import deserialize
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/__init__.py", line 1, in <module>
    from keras.src import activations
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/activations/__init__.py", line 3, in <module>
    from keras.src.activations.activations import elu
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/activations/activations.py", line 1, in <module>
    from keras.src import backend
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/backend/__init__.py", line 10, in <module>
    from keras.src.backend.common.keras_tensor import KerasTensor
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/backend/common/keras_tensor.py", line 2, in <module>
    from keras.src.utils import tree
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keras/src/utils/tree.py", line 12, in <module>
    from tensorflow.python.trackable.data_structures import ListWrapper
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/__init__.py", line 45, in <module>
    from tensorflow._api.v2 import __internal__
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/_api/v2/__internal__/__init__.py", line 8, in <module>
    from tensorflow._api.v2.__internal__ import autograph
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/_api/v2/__internal__/autograph/__init__.py", line 8, in <module>
    from tensorflow.python.autograph.core.ag_ctx import control_status_ctx # line: 34
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/python/autograph/core/ag_ctx.py", line 21, in <module>
    from tensorflow.python.autograph.utils import ag_logging
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/python/autograph/utils/__init__.py", line 17, in <module>
    from tensorflow.python.autograph.utils.context_managers import control_dependency_on_returns
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/python/autograph/utils/context_managers.py", line 19, in <module>
    from tensorflow.python.framework import ops
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 5906, in <module>
    ) -> Optional[Callable[[Any], message.Message]]:
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 243, in inner
    return func(*args, **kwds)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 316, in __getitem__
    return self._getitem(self, parameters)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 433, in Optional
    return Union[arg, type(None)]
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 243, in inner
    return func(*args, **kwds)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 316, in __getitem__
    return self._getitem(self, parameters)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 421, in Union
    parameters = _remove_dups_flatten(parameters)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 215, in _remove_dups_flatten
    all_params = set(params)
TypeError: unhashable type: 'list'

问题原因 #

这个错误是Python 3.9的typing模块与最新版本TensorFlow/Keras的类型注解不兼容导致的。TensorFlow/Keras新版本使用了Python 3.10+支持的灵活类型语法，但Python 3.9的typing模块处理这类语法时会抛出哈希类型错误。

解决方案 #

方案1：升级Python版本到3.10+ #

直接将Python升级到3.10或更高版本，可彻底解决typing模块的兼容性问题，这是最优解。

方案2：降级TensorFlow/Keras到Python3.9兼容版本 #

若不想升级Python，可安装适配Python3.9的旧版依赖：

pip install tensorflow==2.15.0 keras==2.15.0

注意必须保证TensorFlow与Keras版本严格匹配，避免版本冲突。

方案3：替换Keras导入路径为TensorFlow内置版本 #

将代码中的Keras导入语句替换为TensorFlow内置的Keras：

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.optimizers import Adam

部分场景下，tensorflow.keras的兼容性更好，若Keras-RL依赖旧版Keras API，建议配合方案2的版本降级使用。

注意事项 #

• 操作前建议创建虚拟环境，避免污染全局依赖
• 版本调整后，重新安装Keras-RL确保依赖匹配：pip install keras-rl2（原keras-rl已停止维护，推荐使用keras-rl2）

内容的提问来源于stack exchange，提问作者Albifer