在Ansible Tower执行pip3安装命令时触发UnicodeEncodeError错误

问题场景 #

执行以下pip3命令时触发UTF-8编码错误：

pip3 install -r /requirements.txt --no-index --find-links /

错误详情 #

"stderr_lines": 
        "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.",
        "Exception:",
        "Traceback (most recent call last):",
        "  File \"/usr/lib/python3.6/site-packages/pip/basecommand.py\", line 215, in main",
        "    status = self.run(options, args)",
        "  File \"/usr/lib/python3.6/site-packages/pip/commands/install.py\", line 346, in run",
        "    requirement_set.prepare_files(finder)",
        "  File \"/usr/lib/python3.6/site-packages/pip/req/req_set.py\", line 381, in prepare_files",
        "    ignore_dependencies=self.ignore_dependencies))",
        "  File \"/usr/lib/python3.6/site-packages/pip/req/req_set.py\", line 557, in _prepare_file",
        "    require_hashes",
        "  File \"/usr/lib/python3.6/site-packages/pip/req/req_install.py\", line 278, in populate_link",
        "    self.link = finder.find_requirement(self, upgrade)",
        "  File \"/usr/lib/python3.6/site-packages/pip/index.py\", line 465, in find_requirement",
        "    all_candidates = self.find_all_candidates(req.name)",
        "  File \"/usr/lib/python3.6/site-packages/pip/index.py\", line 386, in find_all_candidates",
        "    self.find_links, expand_dir=True)",
        "  File \"/usr/lib/python3.6/site-packages/pip/index.py\", line 236, in _sort_locations",
        "    sort_path(os.path.join(path, item))",
        "  File \"/usr/lib/python3.6/site-packages/pip/index.py\", line 216, in sort_path",
        "    url = path_to_url(path)",
        "  File \"/usr/lib/python3.6/site-packages/pip/download.py\", line 466, in path_to_url",
        "    url = urllib_parse.urljoin('file:', urllib_request.pathname2url(path))",
        "  File \"/usr/lib64/python3.6/urllib/request.py\", line 1689, in pathname2url",
        "    return quote(pathname)",
        "  File \"/usr/lib64/python3.6/urllib/parse.py\", line 891, in quote",
        "    string = string.encode(encoding, errors)",
        "UnicodeEncodeError: 'utf-8' codec can't encode character '\\udcfd' in position 1: surrogates not allowed"

错误核心：无法编码\udcfd代理字符，该字符属于UTF-16代理对，不符合UTF-8编码规范。

排查与解决方法 #

1. 定位问题文件/目录 #

错误源于pip扫描根目录（--find-links /）时，遇到了包含非法代理字符的文件名/目录名。执行以下命令定位异常文件：

find / -maxdepth 1 -print0 | xargs -0 ls -b

该命令会显示转义后的特殊字符，找到包含\udcfd的条目。

2. 临时处理问题文件 #

找到异常文件后，将其移至非pip扫描路径：

mv /[异常文件名] /tmp/

之后重新执行pip安装命令。

3. 优化pip命令路径 #

避免直接扫描根目录，改为指定存放依赖包的具体目录，例如：

pip3 install -r /requirements.txt --no-index --find-links /your-packages-dir/

这样pip仅扫描目标目录，不会触发无关文件的编码问题。

4. 调整系统编码（可选） #

若为系统环境编码异常导致，执行命令前设置UTF-8编码环境变量：

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
pip3 install -r /requirements.txt --no-index --find-links /

内容的提问来源于stack exchange，提问作者Bharath