如何用Python Requests下载指定USGS ZIP文件？

问题 #

需要下载以下地址的ZIP文件：

http://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N45W074.SRTMGL1.hgt.zip

直接在浏览器粘贴该地址可完成下载（浏览器会自动处理内部重定向），但使用Python的requests库尝试自动化下载时，始终遇到连接拒绝错误。

尝试过的代码：

import requests
requests.get(url, 
             allow_redirects=True, 
             headers={'User-Agent':'Chrome/107.0.0.0'})

还试过多种参数组合：使用浏览器完整请求头、设置verify=True、开关重定向、添加HTTPBasicAuth认证（但浏览器无需认证），均出现如下连接错误：

>>> requests.get(url, 
...              allow_redirects=True, 
...              headers={'User-Agent':'Chrome/107.0.0.0'})
Traceback (most recent call last):
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\util\connection.py", line 95, in create_connection
    raise err
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine 
actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\http\client.py", line 1282, in request        
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\http\client.py", line 1328, in _send_request  
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\http\client.py", line 1277, in endheaders     
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\http\client.py", line 1037, in _send_output   
    self.send(msg)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\http\client.py", line 975, in send
    self.connect()
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connection.py", line 205, in connect
    conn = self._new_conn()
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0000016B76A6FE50>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\adapters.py", line 489, in send
    resp = conn.urlopen(
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='e4ftl01.cr.usgs.gov', port=80): Max retries exceeded with url: /MEASURES/SRTMGL1.003/2000.02.11/N45W074.SRTMGL1.hgt.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000016B76A6FE50>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\dere\Miniconda3\envs\blender\lib\site-packages\requests\adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='e4ftl01.cr.usgs.gov', port=80): Max retries exceeded with url: /MEASURES/SRTMGL1.003/2000.02.11/N45W074.SRTMGL1.hgt.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000016B76A6FE50>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))

寻求可成功请求的代码方案（已掌握将响应写入ZIP文件的方法）。

解决方案 #

方案1：改用HTTPS协议 #

目标服务器已不再支持HTTP（80端口）请求，强制要求HTTPS（443端口），替换协议即可解决连接拒绝问题。同时使用标准的User-Agent字符串模拟浏览器请求：

import requests

url = "https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N45W074.SRTMGL1.hgt.zip"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
}

response = requests.get(url, allow_redirects=True, headers=headers)
response.raise_for_status()  # 检查请求是否成功

# 写入文件（示例代码）
with open("N45W074.SRTMGL1.hgt.zip", "wb") as f:
    f.write(response.content)

方案2：使用Session维持会话 #

如果遇到重定向后的会话问题，使用requests.Session可更好地处理Cookie和重定向流程：

import requests

url = "https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N45W074.SRTMGL1.hgt.zip"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
}

session = requests.Session()
session.headers.update(headers)

response = session.get(url, allow_redirects=True)
response.raise_for_status()

with open("N45W074.SRTMGL1.hgt.zip", "wb") as f:
    f.write(response.content)

补充说明 #

• 若网络环境存在代理，需在请求中配置proxies参数，例如：proxies={"https": "http://your-proxy:port"}
• 可添加timeout参数避免请求挂起，如timeout=30（单位：秒）

内容的提问来源于stack exchange，提问作者Derek Eden