通过REST端点流式传输动态压缩文件时内存占用过高

你的代码内存占用过高的核心原因是：

• 默认CopyToAsync使用较大缓冲区，未限制单次读取数据量，导致大量数据堆积在内存中
• HttpClient实例未复用，额外消耗连接池资源
• 未启用分块传输，服务器可能缓存整个响应后再发送

以下是具体优化措施：

1. 修复HttpClient的使用方式 #

构造函数中直接实例化HttpClient会导致频繁创建销毁实例，浪费资源且可能引发内存泄漏，改为依赖注入由框架管理生命周期：

[Route("zip")]    
public class ZipController : ControllerBase
{
    private readonly HttpClient _httpClient;

    public ZipController(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    // 后续方法...
}

2. 限制流复制的缓冲区大小 #

手动指定CopyToAsync的缓冲区大小，控制单次读取的数据量，避免内存暴涨：

// 推荐使用80KB左右的缓冲区，平衡性能与内存占用
const int bufferSize = 81920;
await stream.CopyToAsync(zipStream, bufferSize);

3. 启用响应分块传输 #

在响应头中添加分块编码，让服务器分块发送数据，无需缓存整个压缩包：

Response.Headers.Add("Transfer-Encoding", "chunked");

4. 优化ZipArchive写入逻辑 #

确保每个条目处理完成后及时刷新流，避免内部缓冲堆积：

foreach (var (key, value) in input.FilePathsToUrls)
{
    var zipEntry = zipArchive.CreateEntry(key, CompressionLevel.Optimal);
    await using var zipStream = zipEntry.Open();
    await using var stream = await _httpClient.GetStreamAsync(value);
    
    await stream.CopyToAsync(zipStream, bufferSize);
    await zipStream.FlushAsync();
}
// 最后刷新整个ZipArchive的基础流
await zipArchive.BaseStream.FlushAsync();

完整优化后的代码 #

[Route("zip")]    
public class ZipController : ControllerBase
{
    private readonly HttpClient _httpClient;

    public ZipController(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    [HttpPost]
    public async Task Zip([FromBody] JsonToZipInput input)
    {        
        Response.ContentType = "application/octet-stream";
        Response.Headers.Add($"Content-Disposition", $"attachment; filename=\"{input.FileName}\"");
        // 启用分块传输
        Response.Headers.Add("Transfer-Encoding", "chunked");
        
        using var zipArchive = new ZipArchive(Response.BodyWriter.AsStream(), ZipArchiveMode.Create);
        const int bufferSize = 81920;

        foreach (var (key, value) in input.FilePathsToUrls)
        {
            var zipEntry = zipArchive.CreateEntry(key, CompressionLevel.Optimal);
            await using var zipStream = zipEntry.Open();
            await using var stream = await _httpClient.GetStreamAsync(value);
            
            await stream.CopyToAsync(zipStream, bufferSize);
            await zipStream.FlushAsync();
        }

        await zipArchive.BaseStream.FlushAsync();
    }
}

额外建议 #

• 如果压缩文件数量极多，可控制并发数进行异步并行处理，避免HttpClient连接耗尽
• 监控HttpClient连接池配置，确保不会因并发请求导致资源耗尽

内容的提问来源于stack exchange，提问作者Byresh P