Linux下Python通过代理访问SharePoint请求失败，curl正常求解决

已配置的Linux Shell代理 #

export http_proxy=http://<user_name>:<Pass_word>@<proxy_server>:<proxy_port>
export https_proxy=https://<user_name>:<Pass_word>@<proxy_server>:<proxy_port>

可成功获取令牌的curl命令 #

curl --location --request POST 'https://accounts.accesscontrol.windows.net/{TENANT_ID}/tokens/OAuth/2' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id={CLIENT_ID}@{TENANT_ID}' \
--data-urlencode 'client_secret={CLIENT_SECRET}' \
--data-urlencode 'resource=00000003-0000-0ff1-ce00-000000000000/{tenant}.sharepoint.com@{TENANT_ID}'

失败的Python脚本 #

import requests
from urllib.request import getproxies

def get_access_token():
    print(getproxies()) # 检查是否能读取Linux Shell中设置的代理
    print(type(getproxies())) # <class 'dict'>
    
    headers = {
        "Content-Type": "application/x-www-form-urlencoded",
    }
    body = {
        "grant_type": "client_credentials",
        "client_id": "{CLIENT_ID}@{TENANT_ID}",
        "client_secret": "{CLIENT_SECRET}",
        "resource": "00000003-0000-0ff1-ce00-000000000000/{tenant}.sharepoint.com@{TENANT_ID}",
    }
    _response = requests.post(
        "https://accounts.accesscontrol.windows.net/{TENANT_ID}/tokens/OAuth/2",
        headers=headers,
        data=body,
        proxies=getproxies(), # 尝试过不手动指定代理，均失败
        allow_redirects=False, # 尝试过allow_redirects=True，均失败
    )
    print("REQUEST: ", _response.content)

if __name__ == "__main__":
    get_access_token()

错误信息 #

Traceback (most recent call last):
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/opt/python_3.9.12/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 700, in urlopen
    self._prepare_proxy(conn)
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 994, in _prepare_proxy
    conn.connect()
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f6c6b732be0>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/test_py39/lib/python3.9/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/opt/test_py39/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /{tenant_id}/tokens/OAuth/2 (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6c6b732be0>: Failed to establish a new connection: [Errno -2] Name or service not known')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/projadmin/faker_data/sharep/demo2.py", line 30, in <module>
    _response = requests.post(
  File "/opt/test_py39/lib/python3.9/site-packages/requests/api.py", line 117, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/test_py39/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/test_py39/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/test_py39/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/opt/test_py39/lib/python3.9/site-packages/requests/adapters.py", line 513, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /{tenant_id}/tokens/OAuth/2 (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6c6b732be0>: Failed to establish a new connection: [Errno -2] Name or service not known')))

环境信息 #

• Python 3.9.12
• requests==2.27.1
• Linux CentOS7

解决方法 #

1. 手动构造兼容的代理字典 #

getproxies()返回的代理格式可能不符合requests要求，直接手动构造和curl一致的代理字典：

proxies = {
    "http": "http://<user_name>:<Pass_word>@<proxy_server>:<proxy_port>",
    "https": "http://<user_name>:<Pass_word>@<proxy_server>:<proxy_port>"
}

注意：HTTPS请求的代理协议部分需和curl保持一致，通常用http即可。

2. 验证代理服务器地址可解析性 #

错误提示代理地址无法解析，执行以下命令确认：

ping <proxy_server>
nslookup <proxy_server>

若解析失败，检查系统DNS配置（/etc/resolv.conf），或直接使用代理服务器IP替代主机名。

3. URL编码代理密码中的特殊字符 #

若密码包含@、:、/等特殊字符，curl会自动编码，但requests不会，需手动处理：

from urllib.parse import quote_plus

encoded_password = quote_plus("<Pass_word>")
proxies = {
    "http": f"http://<user_name>:{encoded_password}@<proxy_server>:<proxy_port>",
    "https": f"http://<user_name>:{encoded_password}@<proxy_server>:<proxy_port>"
}

4. 避免代理配置冲突 #

禁止requests自动读取环境变量代理，手动指定构造好的代理：

import os
os.environ.pop('http_proxy', None)
os.environ.pop('https_proxy', None)

# 使用手动构造的proxies发起请求
_response = requests.post(
    "https://accounts.accesscontrol.windows.net/{TENANT_ID}/tokens/OAuth/2",
    headers=headers,
    data=body,
    proxies=proxies,
    allow_redirects=False
)

5. 升级requests版本 #

2.27.1版本存在代理相关已知问题，升级到最新稳定版：

pip install --upgrade requests

内容的提问来源于stack exchange，提问作者code_adithya