You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Temporal Activity触发context deadline exceeded错误排查求助

问题排查:Temporal Activity执行DynamoDB批量写入时触发context deadline exceeded错误

在Kubernetes Pod中运行多个Temporal Workflow,其中某个Workflow启动的Activity会向DynamoDB发起大量BatchWriteItem调用(单次Activity执行近4万次请求)。该Activity的批量写入逻辑代码片段如下:

av, err := attributevalue.MarshalMap(msg)
if err != nil {
    return err
}

writeRequests = append(writeRequests, types.WriteRequest{
    PutRequest: &types.PutRequest{
        Item: av,
    },
})

if len(writeRequests) == maxBatchSize {
    input := &dynamodb.BatchWriteItemInput{
        RequestItems: map[string][]types.WriteRequest{
            repo.tableName: writeRequests,
        },
    }
    out, err := repo.store.BatchWriteItem(ctx, input)
    if err != nil {
        return err
    }
}

执行该Workflow时,Activity正常运行约10分钟后,Pod中突然出现以下错误日志:

2025/03/17 00:02:40 INFO Task processing failed with error Namespace <namespace> TaskQueue <queue-name> WorkerID 1@kubernetes-worker-pod-name@ WorkerType ActivityWorker Error context deadline exceeded

已将Temporal Activity的StartToClose超时设置为2小时,因此该错误并非由此导致。检查Temporal Worker Pod的CPU和内存使用情况均在限制范围内,且DynamoDB控制台中未出现写入请求限流情况。目前无法确定错误原因,寻求排查方向。


更新信息

在Temporal Activity的调用栈中发现以下错误信息:

coroutine root [blocked on chan-2.Receive]:
go.temporal.io/sdk/internal.(*decodeFutureImpl).Get(0xc05ecce318, {0x18f01f8, 0xc0007f0300}, {0x13f11c0, 0xc05ecce180})
/go/src/app/vendor/go.temporal.io/sdk/internal/internal_workflow.go:1588 +0x3e
github.com/twilio-internal/comms-api-broadcast-internal-api/internal/temporal/workflows.IngressWorkflow({0x18f01f8?, 0xc0007f0240?}, {{0xc0001b8150, 0x22}, {0xc0001b8180, 0x2a}, {0xc0007a0060, 0x11}, {0xc00035c557, 0x5}})
/go/src/app/internal/temporal/workflows/ingress.go:44 +0x452
reflect.Value.call({0x1460c60?, 0x1746018?, 0x4158c5?}, {0x16d5a9a, 0x4}, {0xc0007f0270, 0x2, 0xc0007f0270?})
/usr/local/go/src/reflect/value.go:581 +0xca6
reflect.Value.Call({0x1460c60?, 0x1746018?, 0x7fefb9f2c208?}, {0xc0007f0270?, 0x46f49d?, 0x7ff000dfb878?})
/usr/local/go/src/reflect/value.go:365 +0xb9
go.temporal.io/sdk/internal.executeFunction({0x1460c60, 0x1746018}, {0xc0006b0480, 0x2, 0x1466620?})
/go/src/app/vendor/go.temporal.io/sdk/internal/internal_worker.go:1940 +0x26b
go.temporal.io/sdk/internal.(*workflowEnvironmentInterceptor).ExecuteWorkflow(0xc0006c8190, {0x18f01f8, 0xc0007f0210}, 0xc0006a24f8)
/go/src/app/vendor/go.temporal.io/sdk/internal/workflow.go:619 +0x150
go.temporal.io/sdk/interceptor.(*tracingWorkflowInboundInterceptor).ExecuteWorkflow(0xc000517860, {0x18f03f0, 0xc00050a600}, 0xc0006a24f8)
/go/src/app/vendor/go.temporal.io/sdk/interceptor/tracing_interceptor.go:449 +0x2ca
go.temporal.io/sdk/internal.(*workflowExecutor).Execute(0xc0007e2180, {0x18f03f0, 0xc00050a600}, 0xc000802440)
/go/src/app/vendor/go.temporal.io/sdk/internal/internal_worker.go:835 +0x28b
go.temporal.io/sdk/internal.(*syncWorkflowDefinition).Execute.func1({0x18f01f8, 0xc000517920})
/go/src/app/vendor/go.temporal.io/sdk/internal/internal_workflow.go:556 +0xc6

不确定该调用栈在当前场景下的含义。


内容的提问来源于stack exchange,提问作者Sai Krishna

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.13 21:08:14