pip批量升级所有包时出现版本下载异常及AttributeError报错

错误场景 #

执行批量升级本地Python包命令：
pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
触发如下错误流程：

• 系统全局site-packages目录无写入权限，默认切换为用户安装路径
• 检测到absl-py 0.13.0已安装，收集该包时缓存的wheel包因元数据版本不匹配被丢弃
• 下载源码包后执行egg_info命令失败，抛出AttributeError: 'HTMLParser' object has no attribute 'unescape'错误

完整报错日志：

merlins@jimmmmy ~$ pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: absl-py in ./.local/lib/python3.9/site-packages (0.13.0)
Collecting absl-py
  Using cached absl_py-0.13.0-py3-none-any.whl (132 kB)
WARNING: Discarding https://files.pythonhosted.org/packages/23/47/835652c7e19530973c73c65e652fc53bd05725d5a7cf9bb8706777869c1e/absl_py-0.13.0-py3-none-any.whl#sha256=62bd4e248ddb19d81aec8f9446b407ff37c8175c2ba88266a7afa9b4ce4a333b (from https://pypi.org/simple/absl-py/). Requested absl-py from https://files.pythonhosted.org/packages/23/47/835652c7e19530973c73c65e652fc53bd05725d5a7cf9bb8706777869c1e/absl_py-0.13.0-py3-none-any.whl#sha256=62bd4e248ddb19d81aec8f9446b407ff37c8175c2ba88266a7afa9b4ce4a333b has different version in metadata: '0.13.0'
  Downloading absl-py-0.13.0.tar.gz (114 kB)
     |████████████████████████████████| 114 kB 1.1 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zonnvrc9/absl-py_b3a192de46424d63a91fdfb80e09eae8/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zonnvrc9/absl-py_b3a192de46424d63a91fdfb80e09eae8/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-6vtwjr0t
         cwd: /tmp/pip-install-zonnvrc9/absl-py_b3a192de46424d63a91fdfb80e09eae8/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/merlins/.local/lib/python3.9/site-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/home/merlins/.local/lib/python3.9/site-packages/setuptools/dist.py", line 34, in <module>
        from setuptools.depends import Require
      File "/home/merlins/.local/lib/python3.9/site-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/home/merlins/.local/lib/python3.9/site-packages/setuptools/py33compat.py", line 55, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'
    ----------------------------------------

根因分析 #

• 核心问题是本地setuptools版本过低，和Python 3.9版本不兼容：Python 3.9起官方已经完全移除了HTMLParser.unescape接口，旧版本setuptools的兼容逻辑仍尝试调用该废弃接口，触发属性不存在报错。
• 附带的wheel包元数据不匹配警告是absl-py 0.13.0版本的打包缺陷，导致pip放弃使用缓存的wheel包，转而下载源码编译，才触发了setuptools的兼容问题。

解决方案 #

1. 先单独升级setuptools到兼容Python 3.9的版本，适配无全局写入权限的环境，执行命令：
   pip install --upgrade setuptools --user
2. 升级完成后重新执行原始批量升级命令即可：
   pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
3. 若仍存在absl-py的元数据校验报错，可单独跳过缓存安装该版本：
   pip install absl-py==0.13.0 --no-cache-dir --user

内容的提问来源于stack exchange，提问作者Albert Camus Camus