本地运行H2O GPT遇ImportError：无法导入DEFAULT_CIPHERS求解决

问题背景 #

我在conda环境中使用以下命令本地运行H2O GPT，原本运行正常，但更新后出现导入错误：

python3 generate.py \
  --base_model=h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3 \
  -score_model=None \
  --load_4bit=True \
  --prompt_type=human_bot \
  --share=False \
  --gradio_offline_level=1

报错信息 #

Auto set langchain_mode=LLM.  Could use MyData instead.  To allow UserData to pull files from disk, set user_path or langchain_mode_paths, and ensure allow_upload_to_user_data=True
Using Model h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3
Prep: persist_directory=db_dir_UserData exists, using
Starting get_model: h2oai/h2ogpt-gm-oasst1-en-2048-falcon-7b-v3 
Traceback (most recent call last):
  File "/home/user/programs/h2ogpt/generate.py", line 16, in <module>
    entrypoint_main()
  File "/home/user/programs/h2ogpt/generate.py", line 12, in entrypoint_main
    H2O_Fire(main)
  File "/home/user/programs/h2ogpt/src/utils.py", line 60, in H2O_Fire
    fire.Fire(component=component, command=args)
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/user/programs/h2ogpt/src/gen.py", line 1052, in main
    model0, tokenizer0, device = get_model(reward_type=False,
  File "/home/user/programs/h2ogpt/src/gen.py", line 1408, in get_model
    config, _, max_seq_len = get_config(base_model, **config_kwargs, raise_exception=False)
  File "/home/user/programs/h2ogpt/src/gen.py", line 1119, in get_config
    from accelerate import init_empty_weights
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/utils/__init__.py", line 136, in <module>
    from .launch import (
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/utils/launch.py", line 23, in <module>
    from ..commands.config.config_args import SageMakerConfig
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/commands/config/__init__.py", line 19, in <module>
    from .config import config_command_parser
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/commands/config/config.py", line 25, in <module>
    from .sagemaker import get_sagemaker_input
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/accelerate/commands/config/sagemaker.py", line 35, in <module>
    import boto3  # noqa: F401
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/docs/client.py", line 17, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/utils.py", line 37, in <module>
    import botocore.httpsession
  File "/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/botocore/httpsession.py", line 22, in <module>
    from urllib3.util.ssl_ import (
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/home/user/anaconda3/envs/h2ogpt/lib/python3.10/site-packages/urllib3/util/ssl_.py)

核心问题 #

高版本的urllib3移除了DEFAULT_CIPHERS常量，但依赖的botocore库仍在尝试导入该值，导致兼容性错误。

解决方法 #

• 降级urllib3到兼容版本：执行以下命令安装仍包含DEFAULT_CIPHERS的旧版本urllib3

  pip install urllib3==1.26.15
  

• 更新botocore到适配版本：如果不想降级urllib3，尝试更新botocore到支持高版本urllib3的版本

  pip install --upgrade botocore
  

• 重建conda环境：若依赖环境混乱，直接新建干净的conda环境并重新安装H2O GPT依赖

  conda create -n h2ogpt_new python=3.10 -y
  conda activate h2ogpt_new
  git clone https://github.com/h2oai/h2ogpt.git
  cd h2ogpt
  pip install -r requirements.txt
  

• 临时修改导入逻辑（不推荐）：紧急情况下，可手动修改botocore/httpsession.py中的导入代码，将DEFAULT_CIPHERS替换为urllib3当前的等效配置，但此方法仅作为临时 workaround，不建议长期使用。

内容的提问来源于stack exchange，提问作者camilajenny