ASP.NET Core 7.0 Web API写入响应体速度极慢问题求助

AI 小北

2026-7-20

动态生成ZipArchive写入HTTP响应体速度过慢问题

我有一个部署在Server 2016裸机服务器上的ASP.NET Core 7.0 Web API，功能是接收请求时动态生成ZipArchive并返回。从800MB源文件生成300MB的ZipArchive写入内存流仅需30秒，但写入HTTP响应体却耗时约2.5分钟，疑似存在响应体数据缓冲问题。

已尝试的方案/发现

无论使用Kestrel还是IIS，问题均存在
尝试过ControllerBase.File、直接写入响应体、ResponseBody.Writer、Response.BodyWriter.AsStream等多种写入方式
尝试过MemoryStream、BufferedStream（调整缓冲大小会影响下载速度）、FileBufferWriteStream、PipeReader/PipeWriter等流类型
尝试过禁用响应缓冲功能、提高Kestrel最大响应缓冲限制、BinaryWriter、嵌套流等优化手段
服务器性能充足：始终有9GB未使用内存，CPU使用率从未超过20%；若等待所有数据写入响应体后再下载，可达到约300MB/s的带宽峰值
参考Newtonsoft.Json库代码模拟实现，无明显效果

目前核心问题：直接写入响应体的速度仅为写入内存流的1/10；若在响应体未写完时开始下载，速度无法超过20MB/s。已测试启用/禁用分块传输、设置Content-Length的情况，均未解决。

相关代码

控制器代码

[HttpPost("patch")]
[Consumes("text/xml")]
public async Task<IActionResult> FileListAsync()
{
    var languageHeader = Request.Headers["Accept-Language"].ToString();
    var ip = HttpContext.Connection.RemoteIpAddress.ToString();
    string patchFileManifest = "";

    if (patchFileServer.Cache.IsRateLimited(ip))
    {
        logger.LogInformation("Rate limit denying request for {ip}", ip);
        return StatusCode(429, "Rate limit exceeded. Please try again later.");
    }

    using (System.IO.StreamReader reader = new System.IO.StreamReader(Request.Body, Encoding.UTF8))
    {
        patchFileManifest = await reader.ReadToEndAsync();
    }
    logger.LogInformation("Logging request for {ip}", ip);

    return File(patchFileServer.GetPatchFiles(options.Value.RootDirectory, patchFileManifest, languageHeader, ip), "application/zip", $"patch.zip");
}

服务器代码

public Stream GetPatchFiles(string rootPath, string xmlString, string lang, string ip)
{
    try
    {
        ConcurrentBag<PatchFile> filesNeeded = new ConcurrentBag<PatchFile>();
        var clientFiles = XMLParser.ParseXMLFiles(xmlString);
        String clientRootPath = XMLParser.GetRootElement(xmlString).Attribute("SearchPath").Value;
        int rawSize = 0;

        Parallel.ForEach(patchFiles, (patchFile) =>
        {
            if (patchFile.Value.FileInfo.FullName.Contains("Common") || patchFile.Value.FileInfo.FullName.Contains(GetLangString(lang)))
            {
                
                string trimmedPatchFileName = Regex.Match(patchFile.Value.FileInfo.FullName, @"^([^\\]+\\){3}(.*)$").Groups[2].ToString();
                string clientFileFullName = clientRootPath + trimmedPatchFileName;
                string clientFolderName = clientFileFullName.Replace(@"\\" + patchFile.Value.FileInfo.Name, "");
                XElement matchingClientFile = clientFiles.Find(x => x.Attribute("Name").Value.Equals(patchFile.Value.FileInfo.Name) && (x.Parent.Attribute("Name").Value.Equals(clientFolderName) || x.Parent.Attribute("Name").Value.Equals(clientRootPath)));
                if (matchingClientFile == null)
                {
                    filesNeeded.Add(patchFile.Value);
                    rawSize += (int)patchFile.Value.FileInfo.Length / (1024 * 1024);
                }
                else if (!CheckPatchFileSize(patchFile.Value, matchingClientFile.Attribute("SizeByte").Value, ip))
                {
                    filesNeeded.Add(patchFile.Value);
                    rawSize += (int)patchFile.Value.FileInfo.Length / (1024 * 1024);
                }
                else if (!CheckPatchFileDate(patchFile.Value, matchingClientFile.Attribute("ModifiedDate").Value, ip))
                {
                    filesNeeded.Add(patchFile.Value);
                    rawSize += (int)patchFile.Value.FileInfo.Length / (1024 * 1024);
                }
            }
        });
        cache.AddIP(ip, TimeSpan.FromSeconds((int)(rawSize / 20)));
        logger.LogInformation("Raw size of patch request for {ip}: {rawSize}mb rate limited for {timespan}", ip, rawSize, TimeSpan.FromSeconds((int)(rawSize / 20)));
        return patchFileCompressor.GenerateZipFiles(filesNeeded, ip);
    }
    catch (Exception e)
    {
        logger.LogInformation("GetPatchFile error: {error}", e.StackTrace);
        return null;
    }
}

压缩器代码

public Stream GenerateZipFiles(ConcurrentBag<PatchFile> patchFiles, string ip)
{
    Stopwatch stopWatch = new Stopwatch();
    stopWatch.Start();

    var compressedFileStream = new MemoryStream();
    var zipArchive = new ZipArchive(compressedFileStream, ZipArchiveMode.Create, false);

    foreach (var patchFile in patchFiles)
    {
        var zipName = Regex.Match(patchFile.FileInfo.FullName, @"^([^\\]+\\){3}(.*)$").Groups[2];
        var zipEntry = zipArchive.CreateEntry(zipName.ToString());

        using (var originalFileStream = new MemoryStream(patchFile.FileByteArray))
        using (var zipEntryStream = zipEntry.Open())
        {
            originalFileStream.CopyTo(zipEntryStream);
        }
    }
    compressedFileStream.Seek(0, SeekOrigin.Begin);
    stopWatch.Stop();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds,
        ts.Milliseconds / 10);
    logger.LogInformation("Zip size of patch request for {ip}: {zipSize}mb zip time: {zipTime}", ip, compressedFileStream.Length / (1024 * 1024), elapsedTime);
    return compressedFileStream;
}

优化解决方案

方向1：边生成Zip边写入响应体（避免内存缓存全量数据）

当前代码先把整个Zip写入MemoryStream再返回，改成直接将ZipArchive绑定到响应体流，实现流式生成+传输，彻底消除内存缓冲开销：

修改压缩器方法

public async Task GenerateZipFilesToResponse(ConcurrentBag<PatchFile> patchFiles, Stream responseStream, string ip)
{
    Stopwatch stopWatch = new Stopwatch();
    stopWatch.Start();

    // 设置leaveOpen=true，避免ZipArchive关闭响应体流
    using var zipArchive = new ZipArchive(responseStream, ZipArchiveMode.Create, true);

    foreach (var patchFile in patchFiles)
    {
        var zipName = Regex.Match(patchFile.FileInfo.FullName, @"^([^\\]+\\){3}(.*)$").Groups[2].ToString();
        var zipEntry = zipArchive.CreateEntry(zipName);

        // 直接从字节数组写入Zip条目流，避免额外MemoryStream开销
        using var zipEntryStream = zipEntry.Open();
        await zipEntryStream.WriteAsync(patchFile.FileByteArray.AsMemory(0, patchFile.FileByteArray.Length));
    }

    stopWatch.Stop();
    TimeSpan ts = stopWatch.Elapsed;
    string elapsedTime = String.Format("{0:00}:{1:00}:{2:00}.{3:00}",
        ts.Hours, ts.Minutes, ts.Seconds,
        ts.Milliseconds / 10);
    logger.LogInformation("Zip size of patch request for {ip}: {zipSize}mb zip time: {zipTime}", ip, responseStream.Length / (1024 * 1024), elapsedTime);
}

修改控制器逻辑

[HttpPost("patch")]
[Consumes("text/xml")]
public async Task FileListAsync()
{
    var languageHeader = Request.Headers["Accept-Language"].ToString();
    var ip = HttpContext.Connection.RemoteIpAddress.ToString();
    string patchFileManifest = "";

    if (patchFileServer.Cache.IsRateLimited(ip))
    {
        logger.LogInformation("Rate limit denying request for {ip}", ip);
        Response.StatusCode = 429;
        await Response.WriteAsync("Rate limit exceeded. Please try again later.");
        return;
    }

    using (System.IO.StreamReader reader = new System.IO.StreamReader(Request.Body, Encoding.UTF8))
    {
        patchFileManifest = await reader.ReadToEndAsync();
    }
    logger.LogInformation("Logging request for {ip}", ip);

    // 禁用响应缓冲，确保数据实时发送
    Response.Headers.CacheControl = "no-cache";
    Response.Headers.Pragma = "no-cache";
    Response.Headers.Expires = "-1";
    Response.ContentType = "application/zip";
    Response.Headers.ContentDisposition = $"attachment; filename=\"patch.zip\"";
    // 若能提前计算总大小，设置Content-Length可避免分块传输的额外开销
    // Response.ContentLength = CalculateTotalZipSize(filesNeeded);

    // 拆分原GetPatchFiles方法，只返回需要的文件列表（避免生成内存流）
    var filesNeeded = patchFileServer.GetFilesNeeded(options.Value.RootDirectory, patchFileManifest, languageHeader, ip);
    await patchFileCompressor.GenerateZipFilesToResponse(filesNeeded, Response.Body, ip);

    await Response.Body.FlushAsync();
}

方向2：优化内存流返回的传输效率

如果必须先生成完整Zip再返回，可通过以下方式提升传输速度：

配置Kestrel缓冲：在Program.cs中调整Kestrel的响应缓冲参数

builder.WebHost.ConfigureKestrel(options =>
{
    options.Limits.MaxResponseBufferSize = 1024 * 1024 * 64; // 设置64MB缓冲
    options.Limits.MinResponseDataRate = null; // 取消最低传输速率限制
});

使用大缓冲包装流：返回流时用BufferedStream包装，提升写入效率

var memoryStream = patchFileCompressor.GenerateZipFiles(filesNeeded, ip);
return File(new BufferedStream(memoryStream, 65536), "application/zip", "patch.zip");

关键注意事项

流式传输时必须将ZipArchive的leaveOpen参数设为true，否则会关闭响应体流导致异常
提前计算Zip总大小并设置Content-Length，可让客户端提前知晓文件大小，同时避免分块传输的额外开销（计算方式：遍历所有需要的文件，累加ZipArchiveEntry的估算大小）
优化Parallel.ForEach内的文件匹配逻辑，避免重复正则匹配和字符串操作，提升前置处理效率

内容的提问来源于stack exchange，提问作者TDummy

ASP.NET Core 7.0 Web API写入响应体速度极慢问题求助

已尝试的方案/发现 #

相关代码 #

控制器代码 #

服务器代码 #

压缩器代码 #

优化解决方案 #

方向1：边生成Zip边写入响应体（避免内存缓存全量数据） #