使用Python requests向S3分片上传时遭遇SSLError问题求助
问题:通过预签名URL分片上传大文件到S3时出现SSL错误?
我正在编写Python脚本,通过预签名URL将5GB+的大文件上传至S3存储桶。该功能的JavaScript版本可正常运行,因此逻辑和端点应该是有效的。
针对文件的每个分片,我会获取一个预签名分片上传URL,然后尝试向该URL发送PUT请求:
offset = 0 part_number = 0 with open(file_path, 'rb') as f: while offset < file_size_bytes: # 获取当前分片的预签名URL get_multipart_upload_url_params = { "partNumber": part_number, "uploadId": upload_id, "Key": file_key, } get_multipart_upload_url_response = requests.get(GET_MULTIPART_UPLOAD_URL_ENDPOINT, params=get_multipart_upload_url_params) if 'uploadURL' not in get_multipart_upload_url_response.json(): print("Error: Upload Part URL not found in response") sys.exit(1) chunk_upload_url = get_multipart_upload_url_response.json()['uploadURL'] # 上传分片 remaining_bytes = file_size_bytes - offset chunk_size = min(MAX_CHUNK_SIZE, remaining_bytes) chunk = f.read(chunk_size) if not chunk: break response = requests.put(chunk_upload_url, data=chunk) ...
执行requests.put时,遇到如下错误:
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bucket-name.s3.amazonaws.com', port=443): Max retries exceeded with url: [PRESIGNED URL REDACTED] (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2426)')))
更奇怪的是,使用相同接口实现的单分片上传功能完全正常:
# 获取预签名上传URL upload_response = requests.get(SINGLE_PART_UPLOAD_API_ENDPOINT, params={ 'filename': filename, }).json() if 'uploadURL' not in upload_response or 'Key' not in upload_response: print("Error: Upload URL or file key not found in response") sys.exit(1) upload_url = upload_response['uploadURL'] file_key = upload_response['Key'] # 使用requests上传文件 print(f"Uploading: {file_path}") with open(file_path, 'rb') as f: response = requests.put(upload_url, data=f, headers={"Content-Type": "application/octet-stream"}) ...
我已经尝试过以下解决方法:
- 打印预签名分片上传URL并仔细检查其有效性
- 通过浏览器请求确认S3存储桶URL可正常解析
- 更换其他请求库
- 在不同计算机上运行脚本
- 升级OpenSSL、requests和urllib3
内容的提问来源于stack exchange,提问作者Ben K.
相关产品推荐
相关产品推荐

