使用Python requests库下载Word文档时遇ChunkedEncodingError求助

核心原因 #

服务器会校验请求的身份标识（如User-Agent），requests默认请求头与浏览器差异较大，导致服务器中断分块传输；或一次性加载全部内容时，连接不稳定引发中断。

解决方案 #

1. 模拟浏览器请求头 #

复制浏览器的请求头（重点是User-Agent）添加到请求中，让服务器识别为浏览器发起的请求：

import requests

url = 'https://legalref.judiciary.hk/doc/judg/word/vetted/other/ch/2008/CACC000213A_2008.doc'
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
}
res = requests.get(url, headers=headers)
res.raise_for_status()  # 校验请求是否成功
with open('document.doc', 'wb') as f:
    f.write(res.content)

2. 流模式分块下载 #

针对大文件，用stream=True分块读取，避免一次性加载导致连接中断：

import requests

url = 'https://legalref.judiciary.hk/doc/judg/word/vetted/other/ch/2008/CACC000213A_2008.doc'
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
}

with requests.get(url, headers=headers, stream=True) as res:
    res.raise_for_status()
    with open('document.doc', 'wb') as f:
        for chunk in res.iter_content(chunk_size=8192):  # 每次读取8KB
            f.write(chunk)

3. 配置重试机制 #

针对连接不稳定场景，设置HTTP适配器增加重试次数：

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

url = 'https://legalref.judiciary.hk/doc/judg/word/vetted/other/ch/2008/CACC000213A_2008.doc'
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
}

# 配置重试策略
session = requests.Session()
retry = Retry(
    total=3,  # 总重试次数
    backoff_factor=1,  # 重试间隔按1、2、4秒递增
    status_forcelist=[429, 500, 502, 503, 504]  # 需要触发重试的状态码
)
adapter = HTTPAdapter(max_retries=retry)
session.mount('https://', adapter)
session.mount('http://', adapter)

with session.get(url, headers=headers, stream=True) as res:
    res.raise_for_status()
    with open('document.doc', 'wb') as f:
        for chunk in res.iter_content(chunk_size=8192):
            f.write(chunk)

补充说明 #

浏览器能正常下载，是因为它自动携带了符合服务器要求的请求头，且连接中断时会自动重试。以上方法通过模拟浏览器请求行为、分块传输、重试机制，解决服务器中断连接的问题。

内容的提问来源于stack exchange，提问作者lilming