安装transformers==4.27.1时tokenizers编译失败（错误码101）求助

使用ChatGLM-6B需要安装transformers==4.27.1，但执行pip install -v transformers==4.27.1时，tokenizers包编译失败，无法成功构建wheel文件，报错显示cargo rustc命令执行失败并返回错误码101，完整报错信息如下：

pip install -v transformers==4.27.1

....

error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/tokenizers.cpython-312-darwin.so'` failed with code 101
  error: subprocess-exited-with-error
  
  × Building wheel for tokenizers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/dragonfang/****/venv_agi/bin/python3.12 /Users/dragonfang/****/venv_agi/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /var/folders/ll/9dtz3vg150vfv8t75ppq_nr00000gn/T/tmpymy6ke0d
  cwd: /private/var/folders/ll/9dtz3vg150vfv8t75ppq_nr00000gn/T/pip-install-tz2dgt67/tokenizers_11ac58d2069c4ec1985eae0d4528f0ec
  Building wheel for tokenizers (pyproject.toml) ... error
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)

已尝试以下方案但均未解决问题：

• 升级Rust版本至满足clap_lex v0.7.2要求的1.74及以上版本，问题依旧
• 安装无需Rust编译器的transformers==4.34.0，出现AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'错误
• 安装需Rust编译器的transformers==4.33.2，仍触发相同的cargo rustc错误码101问题

现寻求解决该问题的有效方案。

内容的提问来源于stack exchange，提问作者DragonFang