无法直接将BytesIO文件上传至Azure Blob,需先保存至本地硬盘
问题:无法直接将BytesIO对象上传至Azure Blob,需先存本地再读取
用requests从数据服务器获取文件后,直接把BytesIO对象上传到Azure Blob会生成空文件,但先保存到本地硬盘再读取上传就能正常工作。
直接上传(生成空文件)的代码
import BytesIO from azure.storage.blob import BlobServiceClient v_file = BytesIO() for chunk in r.iter_content(chunk_size=1024 * 1024): if chunk: v_file.write(chunk) container_name = 'test' blob_name = 'test_blob' blob_service_client = BlobServiceClient.from_connection_string(connection_string) blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name) blob_client.upload_blob(data=v_file, overwrite=True)
本地保存后上传(正常工作)的代码
download_folder = 'C:/downloaders' fqfilename = download_folder + "\\" + filename with BytesIO() as v_file: for chunk in r.iter_content(chunk_size=1024 * 1024): if chunk: counter += len(chunk) size = counter / (1024) v_file.write(chunk) with open(fqfilename, 'wb') as f: f.write(v_file.getbuffer()) with open (fqfilename, 'rb') as file_to_blob: print(f' {file_to_blob} uploaded to the blob for date {date}') container_name = 'test' blob_name = 'test_blob' blob_service_client = BlobServiceClient.from_connection_string(connection_string) blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name) blob_client.upload_blob(data=v_file, overwrite=True) f.close()
解决方案
问题核心是BytesIO对象的文件指针位置:写完所有数据后,指针停留在文件末尾,upload_blob从当前指针位置开始读取,自然读不到内容。
只需在上传前添加一行代码,把指针重置到文件开头:
v_file.seek(0)
修改后的完整可运行代码:
from io import BytesIO # 修正导入语句,原代码import BytesIO错误 from azure.storage.blob import BlobServiceClient v_file = BytesIO() for chunk in r.iter_content(chunk_size=1024 * 1024): if chunk: v_file.write(chunk) # 关键:重置文件指针到起始位置 v_file.seek(0) container_name = 'test' blob_name = 'test_blob' blob_service_client = BlobServiceClient.from_connection_string(connection_string) blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name) blob_client.upload_blob(data=v_file, overwrite=True)
额外说明:
- 原代码里的
import BytesIO是错误的,Python中BytesIO属于io模块,正确导入方式是from io import BytesIO - 完全不需要把BytesIO内容写入本地文件,重置指针后直接上传就能实现无本地文件的流式上传
内容的提问来源于stack exchange,提问作者Man_Like_Dave
相关产品推荐
相关产品推荐

