Azure WebApp异步代码出现SemaphoreSlim等待致延迟升高问题排查

我们的移动端后端部署于Azure WebApp，近期应用延迟明显升高。通过内存dump分析得到错误提示，提示建议在调用中使用await，但我们的代码已经正确使用了await关键字。

具体报错信息 #

第一条报错 #

Api.Controllers.v5.CampaignsController.Campaigns(string, string) is waiting on System.Threading.SemaphoreSlim.Wait(int, System.Threading.CancellationToken)

对应代码：

public async Task<ActionResult<BaseResponse>> Campaigns([Required] string matasID, string type = "Campaign")
{
    _logger.Information(IntegrationLogger.GetLogMessage(_correlationId, HttpStatusCode.OK,
                                 GetType().Name, IntegrationLogger.GetActualAsyncMethodName(),
                                 $"API Call received with matas Id {matasID} to get compaigns", null));
    if (type == "Campaign")
    {
        var cacheCampaigns = await _memberCacheService.GetCampaigns(matasID);
        BaseResponse response;
        if (cacheCampaigns == null)
        {
            var campaignsResponse = await _clientv5.GetAsync<CampaignsResponseModel>($"/campaigns?type={type}");

            var filteredCampaigns = campaignsResponse.campaigns.Where(c => !string.IsNullOrEmpty(c.link)).ToList();
            campaignsResponse.campaigns = filteredCampaigns; 

            _memberCacheService.SaveCampaigns(campaignsResponse, matasID);
            response = await _memberCacheService.GetCampaigns(matasID);
        }
        else
        {
            response = cacheCampaigns;
        }
        return Ok(response);
    }             
    else
    {
        var responseModel = new FailureResponseModel
        {
            result = false,
            timestamp = DateTime.UtcNow.ToString("yyyyMMddHHmmss"),
            resultDetails = $"Please pass the correct parameters",
            errors = new List<Error>
            {
                new Error{
                description=$"The specified Type '{type}' is not valid",
                code= 400,
                resolve = "Please provide correct parameters"
                }
            }
        };
        return BadRequest(responseModel);
    }
}

以及GetCampaigns方法代码：

public async Task<CampaignsResponseModel> GetCampaigns(string matasID)
{
    try
    {
        _logger.Information(IntegrationLogger.GetLogMessage(_correlationId, HttpStatusCode.OK,
                                 GetType().Name, IntegrationLogger.GetActualAsyncMethodName(),
                                 $"Getting compaigns for User {matasID}", null));

        var cacheCampaigns = await _redisClient.Db3.GetAsync<CampaignsResponseModel>(matasID);

        return cacheCampaigns;
    }
    catch (Exception ex)
    {
        _logger.Error(IntegrationLogger.GetLogMessage(_correlationId, HttpStatusCode.InternalServerError,
                                GetType().Name, IntegrationLogger.GetActualAsyncMethodName(),
                                $"Getting compaigns for User {matasID} Failed", ex.Message));
        throw;
    }
}

第二条警告 #

Api.Controllers.v5.MembersController.GetSpecificMemberInfo(string) is waiting on System.Threading.SemaphoreSlim.Wait(int, System.Threading.CancellationToken)

对应代码：

_logger.Information(IntegrationLogger.GetLogMessage(_correlationId, HttpStatusCode.OK,
                                         GetType().Name, IntegrationLogger.GetActualAsyncMethodName(),
                                         $"API call received to get specific member info with the matas Id {matasID}", null));
    
        MemberShortResponseModel response = await _memberInfoService.GetShortInfo(matasID);
        response.acceptedPermissionVersion = response.termsAcceptVersion;
        Shop shop = null;

        if (response.shopId != null)
        {
            shop = await _shopCacheService.GetShopByShopId(response.shopId).ConfigureAwait(true);
        }

        response.shop = shop;
        return Ok(response);

以及GetShopByShopId方法代码：

public async Task<Shop> GetShopByShopId(string shopId)
{
    _logger.Information(IntegrationLogger.GetLogMessage(_correlationId, HttpStatusCode.OK,
                                 GetType().Name, IntegrationLogger.GetActualAsyncMethodName(),
                                 $"Fetching Shop By shopId", null));

    var cacheShop = await _redisClient.Db1.GetAsync<Shop>(shopId);

    return cacheShop;
}

内存dump截图 #

恳请协助分析此问题的根本原因。

内容的提问来源于stack exchange，提问作者Shabir jan