Python urllib3全局代理配置：ai4bharat-transliteration调用失败

环境信息 #

• 系统：无头Ubuntu 20.04 Server虚拟机
• Python版本：3.8.10
• 网络：企业代理环境，自定义代码中pip、urllib3可正常访问互联网

问题描述 #

按照ai4bharat-transliteration库的说明安装使用，调用e = XlitEngine("hi", beam_width=10, rescore=True)时，库需要联网下载模型文件，但出现urllib3代理连接超时错误，核心报错为代理超时，同时伴随SSL证书未验证警告（推测仅为提示，非核心问题）。已在GitHub提交问题但未获回复，现询问是否可在调用XlitEngine前设置urllib3全局代理，且无需修改该库代码。

错误日志 #

Downloading Multilingual model for transliteration
SSL certificate not verified...
/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '172.xx.yy.zzz'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 700, in urlopen
    self._prepare_proxy(conn)
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 996, in _prepare_proxy
    conn.connect()
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 369, in connect
    self._tunnel()
  File "/usr/lib/python3.8/http/client.py", line 901, in _tunnel
    (version, code, message) = response._read_status()
  File "/usr/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='objects.githubusercontent.com', port=443): Max retries exceeded with url: /github-production-release-asset-2e65be/487173539/4ef3b62d-385b-4a3a-9ab1-a3cc55764ef3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221220%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221220T101158Z&X-Amz-Expires=300&X-Amz-Signature=d24db49d92188df3dbf8a0f1a05126bdaae8bf42289befe734331a41b336f11c&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=487173539&response-content-disposition=attachment%3B%20filename%3Dindicxlit-en-indic-v1.0.zip&response-content-type=application%2Foctet-stream (Caused by ProxyError('Cannot connect to proxy.', timeout('timed out')))

解决方案 #

无需修改库代码，可通过以下几种方式在调用XlitEngine前配置全局代理：

方法1：设置系统环境变量 #

让urllib3/requests自动读取代理配置，代码开头添加：

import os

# 替换为你的代理地址和端口，需认证则格式为 http://用户名:密码@代理地址:端口
os.environ['HTTP_PROXY'] = 'http://172.xx.yy.zzz:端口'
os.environ['HTTPS_PROXY'] = 'http://172.xx.yy.zzz:端口'

# 后续导入并调用XlitEngine
from ai4bharat.transliteration import XlitEngine
e = XlitEngine("hi", beam_width=10, rescore=True)

方法2：替换urllib3全局连接池 #

通过ProxyManager接管所有urllib3请求的代理：

import urllib3
import base64
from ai4bharat.transliteration import XlitEngine

# 创建代理管理器，替换为你的代理地址
proxy = urllib3.ProxyManager(
    'http://172.xx.yy.zzz:端口',
    # 代理需认证时取消注释以下行，替换用户名和密码
    # proxy_headers={'Proxy-Authorization': 'Basic ' + base64.b64encode(b'用户名:密码').decode('utf-8')}
)

# 替换urllib3默认连接池
urllib3.PoolManager = lambda *args, **kwargs: proxy

# 调用XlitEngine
e = XlitEngine("hi", beam_width=10, rescore=True)

方法3：配置requests全局会话代理 #

如果库内部使用requests发送请求，可全局配置会话代理：

import requests
from ai4bharat.transliteration import XlitEngine

# 配置代理字典，替换为你的代理信息
proxy_dict = {
    'http': 'http://172.xx.yy.zzz:端口',
    'https': 'http://172.xx.yy.zzz:端口'
}

# 创建全局会话并设置代理
session = requests.Session()
session.proxies.update(proxy_dict)

# 替换requests默认会话
requests.Session = lambda: session

# 调用XlitEngine
e = XlitEngine("hi", beam_width=10, rescore=True)

补充：消除SSL证书警告 #

若需消除警告（生产环境建议配置企业CA证书，不建议直接禁用），可添加：

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

内容的提问来源于stack exchange，提问作者Gaurav K