使用Cosmos SDK 3.37.0时额外Cosmos DB调用的原因排查

问题背景 #

我使用Cosmos DB执行如下简单检索操作：

public async Task<bool> FindIfDuplicateAsync(string instructionId)
{
    var queryDefinition = new QueryDefinition("Select c.evolveId FROM c where c.instructionId = @instructionId")
        .WithParameter("@instructionId", instructionId);
    var response =  await FindAllItemsAsync<MinimalEvolvePaymentRequest>(queryDefinition, null);
    if (response.Count == 0)
        return false;
    return true;
}

FindAllItemsAsync的定义如下：

protected virtual async Task<List<T>> FindAllItemsAsync<T>(QueryDefinition queryDefinition, QueryRequestOptions requestOptions = null, CancellationToken cancellationToken = default(CancellationToken))
{
    List<T> documents = new List<T>();
    using (FeedIterator<T> feedIterator = Container.GetItemQueryIterator<T>(queryDefinition, null, requestOptions))
    {
        while (feedIterator.HasMoreResults)
        {
            documents.AddRange(await feedIterator.ReadNextAsync(cancellationToken));
        }
    }

    return documents;
}

在Application Insights日志中，发现部分请求耗时长达数秒，包含大量额外GET调用：

https://{{cosmosResource}}.documents.azure.com:443/
http://169.254.169.254/metadata/instance?api-version=2020-06-01
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/paymentRequests
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/paymentRequests/docs
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/WXteAA==/colls/WXteAIy5we0=/pkranges
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/WXteAA==/colls/WXteAIy5we0=/pkranges
https://{{cosmosResource}}-eastus.documents.azure.com//addresses/?$resolveFor=dbs%2fWXteAA%3d%3d%2fcolls%2fWXteAIy5we0%3d%2fdocs&$filter=protocol eq rntbd&$partitionKeyRangeIds=0
https://{{cosmosResource}}.documents.azure.com:443/
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/partnerLedger
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/partnerLedger/docs
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/WXteAA==/colls/WXteALWILLw=/pkranges
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/WXteAA==/colls/WXteALWILLw=/pkranges
https://{{cosmosResource}}-eastus.documents.azure.com//addresses/?$resolveFor=dbs%2fWXteAA%3d%3d%2fcolls%2fWXteALWILLw%3d%2fdocs&$filter=protocol eq rntbd&$partitionKeyRangeIds=0
https://{{cosmosResource}}.documents.azure.com:443/
...（重复的Cosmos DB端点调用）

而常规请求的日志仅包含：

https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/paymentRequests/docs
https://{{cosmosResource}}-eastus.documents.azure.com/dbs/payments/colls/partnerLedger/docs

注：文档使用非分层分区键，无RU节流，CosmosClient通过自定义NuGet包构建，代码如下：

protected virtual CosmosClient CosmosClient
{
    get
    {
        if (_cosmosClient != null)
        {
            return _cosmosClient;
        }

        CosmosClientBuilder cosmosClientBuilder = GetCosmosClientBuilder();

        cosmosClientBuilder = cosmosClientBuilder.WithApplicationName(ApplicationName);
        cosmosClientBuilder = cosmosClientBuilder.WithApplicationPreferredRegions(ApplicationPreferredRegions);
        cosmosClientBuilder = cosmosClientBuilder.WithApplicationRegion(ApplicationRegion);

        if (HttpClientFactory is not null)
            cosmosClientBuilder = cosmosClientBuilder.WithHttpClientFactory(() => HttpClientFactory.CreateClient());

        if (AllowBulkExecution.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithBulkExecution(AllowBulkExecution.Value);

        if (ConsistencyLevel.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithConsistencyLevel(ConsistencyLevel.Value);

        // Don't set the CosmosSerializer unless it's not null (the setter has validation)
        if (CosmosSerializer != null)
            cosmosClientBuilder = cosmosClientBuilder.WithCustomSerializer(CosmosSerializer);

        // Don't set the CosmosSerializationOptions unless it's not null (the setter has validation)
        if (CosmosSerializationOptions != null)
            cosmosClientBuilder = cosmosClientBuilder.WithSerializerOptions(CosmosSerializationOptions);

        if (EnableContentResponseOnWrite.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithContentResponseOnWrite(EnableContentResponseOnWrite.Value);

        if (LimitToEndpoint.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithLimitToEndpoint(LimitToEndpoint.Value);

        if (RequestTimeout.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithRequestTimeout(RequestTimeout.Value);

        if (MaxRetryWaitTimeOnThrottledRequests.HasValue && MaxRetryAttemptsOnThrottledRequests.HasValue)
            cosmosClientBuilder = cosmosClientBuilder.WithThrottlingRetryOptions(MaxRetryWaitTimeOnThrottledRequests.Value, MaxRetryAttemptsOnThrottledRequests.Value);

        cosmosClientBuilder = cosmosClientBuilder.WithConnectionModeDirect();

        _cosmosClient = cosmosClientBuilder.Build();

        return _cosmosClient;
    }
}

额外GET请求的来源与原因 #

1. 元数据与地址解析类请求 #

• http://169.254.169.254/metadata/instance：这是Azure VM实例元数据服务的调用，SDK通过它获取当前实例的区域信息，用于匹配Cosmos DB的首选区域，确保请求路由到最近的可用节点。
• /dbs/xxx/colls/xxx：SDK首次访问容器时，会获取容器的元数据（如分区键定义、索引策略等），后续请求会缓存这些数据，但缓存失效或首次访问该容器时，会触发此调用。
• /pkranges：用于获取容器的分区键范围信息，Direct连接模式下，SDK需要知道每个分区对应的物理节点地址才能直接路由请求，分区范围变更或缓存过期时会重复调用。
• /addresses/?$resolveFor=...：这是Direct模式下的地址解析请求，SDK通过它获取特定分区键范围对应的物理节点RNTBD（TCP协议）地址，用于建立直接连接。连接断开、节点变更或缓存失效时会触发此请求。

2. 重复调用的触发场景 #

• CosmosClient未全局复用：CosmosClient是线程安全的，应全局复用单例。如果你的代码中频繁实例化包含该CosmosClient属性的类，会导致多次创建CosmosClient实例，重复执行初始化操作。
• 缓存失效或未命中：SDK会缓存容器元数据、分区范围和节点地址，但缓存有过期时间，或集群拓扑变化（如节点故障、扩容）时缓存失效，SDK需要重新获取这些信息。
• Direct连接模式特性：你使用了WithConnectionModeDirect()，这种模式下SDK需要直接与物理节点通信，因此必须获取分区范围和节点地址，相比Gateway模式会有更多初始化元数据请求，但后续请求性能更优。

3. 优化建议 #

• 确保CosmosClient单例复用：检查代码中是否存在频繁实例化CosmosClient所属类的情况，确保全局仅存在一个CosmosClient实例。
• 提前预热元数据：在应用启动时，提前访问所有需要使用的容器，触发元数据缓存，避免首次业务请求时执行初始化操作。
• 验证区域配置：确保WithApplicationPreferredRegions和WithApplicationRegion配置正确，避免SDK频繁尝试切换区域导致的地址解析请求。

内容的提问来源于stack exchange，提问作者MAK