.NET 8请求解压中间件gzip编码InvalidDataException问题排查

目标框架：netstandard2.0，同时在*.net8*环境下测试
操作系统：Windows 10
请求由Windows应用中的HttpClient发起，源代码如下：

Test(someHostName, someSecretePath, jsonContent);

private static void Test(string host, string url, string jsonContent)
{
    var client = new HttpClient()
    {
        BaseAddress = new Uri(host)
    };
    var request = new HttpRequestMessage
    {
        Method = new HttpMethod("Post"),
        RequestUri = new Uri(url),
    };
    request.Content = new StringContent(Compress(jsonContent), Encoding.UTF8, "application/json");
    request.Content.Headers.ContentEncoding.Add("gzip");

    using var result = client.Send(request);
}

private static string Compress(string inputStr)
{
    var bytes = Encoding.Unicode.GetBytes(inputStr);
    using (var msi = new MemoryStream(bytes))
    {
        using (var mso = new MemoryStream())
        {
            using (var gs = new GZipStream(mso, CompressionMode.Compress, true))
            {
                msi.CopyTo(gs);
            }
            return Convert.ToBase64String(mso.ToArray());
        }
    }
}

目标框架：.net8.0
托管方式：Kestrel
Program.cs代码如下：

public static void Main(string[] args)
{
    var builder = WebApplication.CreateBuilder(args);

    // Add services to the container.
    builder.Services.AddRequestDecompression();
    builder.Services.AddEndpointsApiExplorer();
    builder.Services.AddControllers().AddNewtonsoftJson();

    var app = builder.Build();

    app.UseRequestDecompression();
    app.MapControllers();
    app.Run();
}

HTTP请求详情 #

Protocol: HTTP/1.1
Method: POST
Scheme: http
PathBase: 
Path: /someSecretPath
Host: someSecretHost
Content-Type: application/json; charset=utf-8
Content-Length: 236
Content-Encoding: gzip

调试窗口日志 #

dbug: Microsoft.AspNetCore.RequestDecompression.DefaultRequestDecompressionProvider[4]
      **The request will be decompressed with 'gzip'**.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'WebApiNet8.Controllers.SomeController.CreateAsync (WebApi)'
dbug: Microsoft.AspNetCore.Server.Kestrel[25]
      Connection id "0HN11D3HP3G05", Request id "0HN11D3HP3G05:00000001": started reading request body.
dbug: Microsoft.AspNetCore.Server.Kestrel[26]
      Connection id "0HN11D3HP3G05", Request id "0HN11D3HP3G05:00000001": done reading request body.
Exception thrown: "System.IO.InvalidDataException" в System.Private.CoreLib.dll

fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.IO.InvalidDataException: **The archive entry was compressed using an unsupported compression method.**
         at System.IO.Compression.Inflater.Inflate(FlushCode flushCode)
         at System.IO.Compression.Inflater.ReadInflateOutput(Byte* bufPtr, Int32 length, FlushCode flushCode, Int32& bytesRead)
         at System.IO.Compression.Inflater.ReadOutput(Byte* bufPtr, Int32 length, Int32& bytesRead)
         at System.IO.Compression.Inflater.InflateVerified(Byte* bufPtr, Int32 length)
         at System.IO.Compression.DeflateStream.<ReadAsyncMemory>g__Core|51_0(Memory`1 buffer, CancellationToken cancellationToken)
         at SizeLimitedStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)

从日志中可以看到，解压中间件已找到合适的解压提供程序并创建了流，但在读取流时出现数据读取错误。

请问是什么原因导致该问题？该如何避免？

原因分析 #

问题出在客户端的压缩逻辑：

1. 你先用GZipStream把字符串的Unicode字节压缩成二进制流，随后将这个二进制流Base64编码为字符串，最后通过StringContent把Base64字符串以UTF-8编码发送给服务端。
2. 服务端的RequestDecompression中间件会直接将请求体字节当作标准gzip压缩数据解压，但实际收到的是Base64编码后的字节，并非合法的gzip二进制数据，因此抛出"不支持的压缩方法"异常。

解决方案 #

修改客户端代码，直接发送gzip压缩后的二进制数据，跳过Base64编码步骤：

1. 将Compress方法的返回类型改为byte[]，直接返回压缩后的二进制数组。
2. 使用ByteArrayContent替代StringContent封装请求体，避免额外的编码转换。

修改后的客户端代码示例：

Test(someHostName, someSecretePath, jsonContent);

private static void Test(string host, string url, string jsonContent)
{
    var client = new HttpClient()
    {
        BaseAddress = new Uri(host)
    };
    var request = new HttpRequestMessage
    {
        Method = HttpMethod.Post,
        RequestUri = new Uri(url),
    };
    // 用ByteArrayContent直接传递压缩后的二进制数据
    request.Content = new ByteArrayContent(Compress(jsonContent));
    request.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
    request.Content.Headers.ContentEncoding.Add("gzip");

    using var result = client.Send(request);
}

private static byte[] Compress(string inputStr)
{
    // 改用UTF8编码，和服务端接收编码一致，减少数据体积
    var bytes = Encoding.UTF8.GetBytes(inputStr);
    using (var msi = new MemoryStream(bytes))
    {
        using (var mso = new MemoryStream())
        {
            using (var gs = new GZipStream(mso, CompressionMode.Compress, true))
            {
                msi.CopyTo(gs);
            }
            return mso.ToArray();
        }
    }
}

额外优化点：

• 原代码使用Encoding.Unicode（UTF-16）编码字符串，改成Encoding.UTF8可减少数据体积，同时避免编码不一致问题。
• 直接使用HttpMethod.Post静态属性，无需实例化新的HttpMethod对象。

内容的提问来源于stack exchange，提问作者NikolaDo