FastAPI UploadFile与aiohttp多部分POST请求兼容问题排查

我在FastAPI中实现了一个接收图片上传的API，需要将接收到的图片通过multipart/form-data转发给外部API，初始代码如下：

import aiohttp


@app.post(
    "/images",
    status_code=status.HTTP_201_CREATED,
)
async def upload_image(
    image: UploadFile,
):
    url = 'some.api.com'
    async with aiohttp.ClientSession(raise_for_status=True) as session:
        data = aiohttp.FormData()
        data.add_field("file", image.file)
        async with session.post(url, data=data) as response:
            r = await response.json()
    return r.json()

错误信息 #

运行后触发如下类型错误：

File "/usr/local/lib/python3.10/site-packages/aiohttp/formdata.py", line 149, in _gen_form_data
raise TypeError(
TypeError: Can not serialize value type: <class 'tempfile.SpooledTemporaryFile'>

根据FastAPI官方文档说明：

file: A SpooledTemporaryFile (a file-like object). This is the actual Python file that you can pass directly to other functions or libraries that expect a "file-like" object.

但aiohttp并不支持直接传递这种同步临时文件对象。

已尝试的无效方法 #

• 读取全部文件字节后传递：通过img_bytes = await image.read()读取字节，再用data.add_field("file", img_bytes, filename='name', content_type='image/jpeg')传入aiohttp，虽可能生效但会将整个文件加载到内存，违背了使用SpooledTemporaryFile的初衷
• 直接传入UploadFile对象image而非image.file，无效
• 传入image.file时附加filename='name', content_type='image/jpeg'参数，无效
• 使用aiohttp.MultipartWriter，代码如下，但仍报错TypeError: Cannot create payload from <tempfile.SpooledTemporaryFile object>：

async with aiohttp.ClientSession(raise_for_status=True) as session:
    with aiohttp.MultipartWriter('form-data') as data:
        part = data.append(image.file)
        part.set_content_disposition('form-data', name='file', filename='test.jpeg')
        async with session.post(url, data=data) as response:
            r = await response.json()

解决方案 #

问题根源在于aiohttp无法直接处理同步的SpooledTemporaryFile对象，需要将其包装为异步可迭代的流对象，同时保持分块传输以避免内存溢出。修改后的代码如下：

import aiohttp
from fastapi import UploadFile, status
from aiohttp import StreamReader
import asyncio


@app.post(
    "/images",
    status_code=status.HTTP_201_CREATED,
)
async def upload_image(
    image: UploadFile,
):
    url = 'some.api.com'
    # 重置文件指针到开头，确保读取完整内容
    await image.seek(0)
    
    # 定义异步分块读取函数，在线程池中处理同步文件读取
    async def read_file_chunks():
        while chunk := await asyncio.to_thread(image.file.read, 8192):
            yield chunk
    
    # 将分块读取器包装为aiohttp可处理的StreamReader
    async_stream = StreamReader(read_file_chunks())
    
    async with aiohttp.ClientSession(raise_for_status=True) as session:
        data = aiohttp.FormData()
        data.add_field(
            "file",
            async_stream,
            filename=image.filename,
            content_type=image.content_type
        )
        async with session.post(url, data=data) as response:
            response_data = await response.json()
    return response_data

说明 #

1. await image.seek(0)：确保文件指针回到起始位置，避免因FastAPI内部读取导致后续读取为空
2. asyncio.to_thread：将同步的文件读取操作放到线程池中执行，不阻塞异步事件循环
3. StreamReader：将分块读取的内容包装为aiohttp可识别的异步流对象，实现分块传输，避免一次性加载整个文件到内存

内容的提问来源于stack exchange，提问作者JKJK