Azure Cosmos DB点读延迟未达SLA标准，求排查方案

问题概述 #

线上ASP.NET Core API使用Azure Cosmos DB时，点读延迟稳定在20-25ms，未达到SLA规定的P99延迟<10ms、中位数4-5ms的水平。本地测试点读平均2-3ms（最低<1ms），且线上写入延迟均低于10ms，仅读操作存在明显延迟问题。

环境配置 #

• 区域：北欧区域（应用服务与Cosmos DB同区域部署）
• 应用服务：P1mV3 Windows应用服务（代码部署）
• Cosmos DB：Serverless模式NoSQL
• SDK：.NET 8 最新Cosmos客户端
• 连接设置：Direct连接模式，禁用批量执行，单点点读成本为1RU

测试代码 #

using Azure.Monitor.OpenTelemetry.AspNetCore;
using Microsoft.Azure.Cosmos;
using System.Diagnostics;
namespace CosmosTest
{
    public class Program
    {
        static string? _dbId;
        static string? _containerId;
        public static void Main(string[] args)
        {
            var builder = WebApplication.CreateBuilder(args);

            builder.Services.AddSingleton(serviceProvider =>
            {
                var configuration = serviceProvider.GetRequiredService<IConfiguration>();

                _dbId = configuration["CosmosDB:DatabaseId"];
                _containerId = configuration["CosmosDB:ContainerId"];
                var cosmosClientOptions = new CosmosClientOptions
                {
                    SerializerOptions = new CosmosSerializationOptions
                    {
                        PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase
                    },
                    ConnectionMode = ConnectionMode.Direct,

                    AllowBulkExecution = false,
                };
                var connectionString = configuration.GetConnectionString("Cosmos");
                return new CosmosClient(connectionString);
            });

            builder.Services.AddOpenTelemetry().WithTracing(tracerProviderBuilder =>
            {
                tracerProviderBuilder.AddSource("Azure.Cosmos.Operation");

            }).UseAzureMonitor();

            builder.Services.AddApplicationInsightsTelemetry();
            builder.Logging.AddApplicationInsights();

            var app = builder.Build();

            app.MapGet("/{partitionKey}/{id}", async (Guid id, Guid partitionKey, CosmosClient cosmosClient, ILogger<Program> logger) =>
            {
                var container = cosmosClient.GetContainer(_dbId, _containerId);

                var startTime = Stopwatch.GetTimestamp();

                var itemResponse = await container.ReadItemAsync<Product>(id.ToString(), new PartitionKey(partitionKey.ToString()));

                var elapsed = Stopwatch.GetElapsedTime(startTime);

                logger.LogInformation("Read item {id} from container {container} in {elapsed} ms", id, _containerId, elapsed);

                return Results.Ok(itemResponse.Resource);

            });

            app.MapGet("/{partitionKey}", async (Guid partitionKey, CosmosClient cosmosClient) =>
            {
                var container = cosmosClient.GetContainer(_dbId, _containerId);
                var query = new QueryDefinition("SELECT * FROM c WHERE c.productBlueprintId = @productBlueprintId")
                    .WithParameter("@productBlueprintId", partitionKey);
                var iterator = container.GetItemQueryIterator<Product>(query);
                var results = new List<Product>();
                while (iterator.HasMoreResults)
                {
                    var response = await iterator.ReadNextAsync();
                    results.AddRange(response.ToList());
                }
                return Results.Ok(results);
            });

            app.MapPost("/", async (Product product, CosmosClient cosmosClient) =>
            {
                var container = cosmosClient.GetContainer(_dbId, _containerId);
                var response = await container.CreateItemAsync(product, new PartitionKey(product.productBlueprintId.ToString()));
                return;
            });

            app.UseDeveloperExceptionPage();

            app.Run();
        }
    }

    public record Product(string id, Guid productBlueprintId, string Name, Guid OwnerId, DateTime createdUtc, Guid createdBy);
}

排查方向与优化建议 #

1. 客户端连接预热 #

线上首次请求会触发Cosmos Client的连接池初始化，导致延迟偏高。可在应用启动时主动执行预热操作：

// 在app.Build()之后添加
using var scope = app.Services.CreateScope();
var cosmosClient = scope.ServiceProvider.GetRequiredService<CosmosClient>();
var container = cosmosClient.GetContainer(_dbId, _containerId);
// 执行测试点读以预热连接
try
{
    await container.ReadItemAsync<Product>("warmup-id", new PartitionKey("warmup-pk"));
}
catch { } // 忽略测试请求的错误

2. 规避Serverless冷启动 #

Cosmos DB Serverless模式在闲置一段时间后会进入冷状态，首次请求需唤醒容器，导致延迟上升。可通过定期暖场请求（例如每分钟执行一次点读）保持容器活跃，避免冷启动。

3. 网络配置排查 #

• 若应用服务启用了VNet集成，检查VNet到Cosmos DB的网络路径是否存在额外延迟；
• 若未使用Private Endpoint，确认公共网络访问的路由是否为最优路径；

4. 客户端配置优化 #

• 启用TCP Keep-Alive，避免连接被中断后重新建立的开销：

  var cosmosClientOptions = new CosmosClientOptions
  {
      // 现有配置...
      ConnectionMode = ConnectionMode.Direct,
      TcpKeepAliveInterval = TimeSpan.FromMinutes(1),
      TcpKeepAliveTime = TimeSpan.FromMinutes(5)
  };
  

• 检查序列化开销：确认Product记录的属性映射是否合理，避免不必要的序列化/反序列化耗时。

5. Cosmos DB端监控分析 #

• 在Azure Portal查看Cosmos DB的Metrics，筛选"Total Request Latency"和"Server Side Latency"指标，区分客户端总延迟与服务端实际处理延迟；
• 检查分区热点：查看"Partition Key Statistics"，确认是否存在某个分区RU使用率过高导致的延迟上升；

6. 应用服务资源排查 #

• 监控P1mV3实例的CPU、内存使用率，若资源不足会导致请求处理排队，增加延迟；
• 确认应用服务的缩放配置，确保实例数能承载当前请求量。

内容的提问来源于stack exchange，提问作者Erik83