ASP.NET Core高负载下JsonSerializer.DeserializeAsync调用失败问题

问题描述 #

在高负载场景下，ASP.NET Core应用调用System.Text.Json.JsonSerializer.DeserializeAsync时频繁失败，并发请求增多时会抛出JSON转换错误，应用响应速度急剧下降。

错误信息 #

fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.Text.Json.JsonException: The JSON value could not be converted to System.Collections.Generic.List`1[ServiceContract.Models.City]. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
         at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
         at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
         at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
         at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
         at CitiesService.CovidService.GetCities() in C:\Users\ahmed\RiderProjects\CitiesService\CovidService.cs:line 28
         at HttpClientExample.Controllers.HomeController.Home() in C:\Users\ahmed\RiderProjects\HttpClientExample\Controllers\HomeController.cs:line 19
         at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

项目结构 #

ClientService
  - CovidService.cs
  - CovidServiceSettings.cs
HttpClientExample (主项目)
ServiceContract
  - Models
    - City.cs
  - ICovidService.cs

现有代码 #

CovidService.cs #

using System.Text.Json;
using ServiceContract.Models;
using ServiceContract;

namespace CitiesService;

public class CovidService : ICovidService
{
    private readonly HttpClient _httpClient;
    private readonly CovidServiceSettings _covidServiceSettings;

    public CovidService(HttpClient httpClient, CovidServiceSettings covidServiceSettings)
    {
        _httpClient = httpClient;
        _covidServiceSettings = covidServiceSettings;
    }
    
    public async Task<List<City>> GetCities()
    {
        HttpRequestMessage requestMessage = new()
        {
            RequestUri = new Uri(_covidServiceSettings.Url),
            Method = HttpMethod.Get
        };

        HttpResponseMessage responseMessage = await _httpClient.SendAsync(requestMessage);
        Stream stream = await responseMessage.Content.ReadAsStreamAsync();
        List<City> cities = await JsonSerializer.DeserializeAsync<List<City>>(stream);
        return cities;
    }
}

Program.cs #

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();
builder.Services.AddHttpClient();
builder.Services.AddScoped<ICovidService, CovidService>(serviceProvider =>
{
    var httpClientFactory = serviceProvider.GetService<IHttpClientFactory>();
    var configuration = serviceProvider.GetService<IConfiguration>();

    return new CovidService(httpClientFactory!.CreateClient("CovidService"),
        configuration.GetSection("CovidSettings").Get<CovidServiceSettings>());
});

var app = builder.Build();
app.UseStaticFiles();
app.UseRouting();
app.MapControllers();
app.Run();

HomeController.cs #

using Microsoft.AspNetCore.Mvc;
using ServiceContract;

namespace HttpClientExample.Controllers;

public class HomeController : Controller
{
    private readonly ICovidService _covidService;

    public HomeController(ICovidService covidService)
    {
        _covidService = covidService;
    }

    [Route("/")]
    public async Task<IActionResult> Home()
    {
        var cities = await _covidService.GetCities();
        return View(cities);
    }
}

已尝试操作 #

调整ThreadPool.SetMinThreads提升线程池限制，高负载下性能略有提升，能成功处理前20-30个请求，但后续响应迅速恶化，瓶颈仍未解决。

预期目标 #

JsonSerializer.DeserializeAsync在高并发请求下能稳定运行，无明显性能下降或转换失败。

问题分析与解决方案 #

核心问题点 #

1. HttpClient管理不当：手动创建HttpClient实例未利用连接池优化，高负载下易出现连接耗尽。
2. JsonSerializerOptions未复用：每次反序列化创建新实例，造成内存与性能损耗。
3. 缺乏响应校验：未验证上游服务的响应状态与内容格式，错误内容直接引发反序列化失败。
4. 无重试机制：上游服务波动时无重试逻辑，单次失败直接导致应用错误。

具体修复方案 #

1. 使用Typed Client优化HttpClient生命周期

将CovidService注册为Typed Client，由IHttpClientFactory自动管理连接池：

修改Program.cs：

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();

// 注册Typed Client并绑定配置
builder.Services.AddHttpClient<ICovidService, CovidService>((sp, client) =>
{
    var settings = sp.GetRequiredService<IConfiguration>()
        .GetSection("CovidSettings").Get<CovidServiceSettings>();
    client.BaseAddress = new Uri(settings.Url);
});

var app = builder.Build();
app.UseStaticFiles();
app.UseRouting();
app.MapControllers();
app.Run();

修改CovidService.cs构造函数：

public class CovidService : ICovidService
{
    private readonly HttpClient _httpClient;

    public CovidService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }
    
    public async Task<List<City>> GetCities()
    {
        using HttpResponseMessage response = await _httpClient.GetAsync(string.Empty);
        response.EnsureSuccessStatusCode();
        
        using Stream stream = await response.Content.ReadAsStreamAsync();
        return await JsonSerializer.DeserializeAsync<List<City>>(stream, JsonSerializerOptionsCache.Default) 
               ?? new List<City>();
    }
}

2. 复用JsonSerializerOptions

创建静态缓存类避免重复实例化：

namespace CitiesService;

public static class JsonSerializerOptionsCache
{
    public static readonly JsonSerializerOptions Default = new()
    {
        PropertyNameCaseInsensitive = true,
        // 添加其他自定义配置
    };
}

3. 添加Polly重试策略

安装依赖包：

dotnet add package Polly
dotnet add package Microsoft.Extensions.Http.Polly

修改Program.cs的HttpClient注册：

builder.Services.AddHttpClient<ICovidService, CovidService>((sp, client) =>
{
    var settings = sp.GetRequiredService<IConfiguration>()
        .GetSection("CovidSettings").Get<CovidServiceSettings>();
    client.BaseAddress = new Uri(settings.Url);
})
// 指数退避重试3次
.AddPolicyHandler(Policy
    .Handle<HttpRequestException>()
    .OrResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
    .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt))));

4. 增强响应校验

在反序列化前验证内容类型：

public async Task<List<City>> GetCities()
{
    using HttpResponseMessage response = await _httpClient.GetAsync(string.Empty);
    response.EnsureSuccessStatusCode();
    
    var contentType = response.Content.Headers.ContentType?.MediaType;
    if (string.IsNullOrEmpty(contentType) || !contentType.Equals("application/json", StringComparison.OrdinalIgnoreCase))
    {
        throw new InvalidOperationException("上游服务返回非JSON格式内容");
    }
    
    using Stream stream = await response.Content.ReadAsStreamAsync();
    return await JsonSerializer.DeserializeAsync<List<City>>(stream, JsonSerializerOptionsCache.Default) 
           ?? new List<City>();
}

内容的提问来源于stack exchange，提问作者Wayfaring Tenno