使用AWS Bedrock调用Sonnet 3.5时持续触发Throttling Exception求助

在AWS Bedrock使用Claude 3.5 Sonnet（anthropic.claude-3-5-sonnet-20240620-v1:0）模型时，首次调用成功消耗约1800 tokens，但所有后续调用均触发ThrottlingException。已知该模型公开限额为每分钟400,000 tokens，当前调用量远未触及该阈值。

首次调用（成功）日志 #

{
    "schemaType": "ModelInvocationLog",
    "schemaVersion": "1.0",
    "timestamp": "2025-02-07T08:11:59Z",
    "accountId": "821052193763",
    "identity": {
        "arn": "arn:aws:iam::821052193763:user/aws_demos"
    },
    "region": "us-east-1",
    "requestId": "ea303aba-9b49-4d9e-bd25-4d839878a1f8",
    "operation": "ConverseStream",
    "modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "input": {
        "inputContentType": "application/json",
        "inputBodyJson": {
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {
                            "text": "Use all the tools at your disposal to find the right answer for the query below: \n\nWhat is the favorability rating on work-life balance and flexibility for Company1 in the year 2022??\n\n\n (reminder to respond in a JSON blob no matter what)"
                        }
                    ]
                }
            ],
            "system": [
                {
                    "text": "Respond to the human as helpfully and accurately as possible. You have access to the following tools:\n\nVectorStore(*args: Any, callbacks: Union[List[langchain_core.callbacks.base.BaseCallbackHandler], langchain_core.callbacks.base.BaseCallbackManager, NoneType] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) -> Any - Useful for searching information from the knowledge base. This should be given the first priority while searching for information. It has information about Company1, Company2 and Company3's ESG reports in detail. If the information is not found, then the database tools must be used to find the answer, args: {'tool_input': {'type': 'string'}}\nsql_db_query - Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields., args: {'query': {'title': 'Query', 'description': 'A detailed and correct SQL query.', 'type': 'string'}}\nsql_db_schema - Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3, args: {'table_names': {'title': 'Table Names', 'description': \"A comma-separated list of the table names for which to return the schema. Example input: 'table1, table2, table3'\", 'type': 'string'}}\nsql_db_list_tables - Input is an empty string, output is a comma-separated list of tables in the database., args: {'tool_input': {'title': 'Tool Input', 'description': 'An empty string', 'default': '', 'type': 'string'}}\nsql_db_query_checker - Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!, args: {'query': {'title': 'Query', 'description': 'A detailed and SQL query to be checked.', 'type': 'string'}}\n\nUse a json blob to specify a tool by providing an action key (tool name) and an action_input key (tool input).\n\nValid \"action\" values: \"Final Answer\" or VectorStore, sql_db_query, sql_db_schema, sql_db_list_tables, sql_db_query_checker\n\nProvide only ONE action per $JSON_BLOB, as shown:\n\n```\n{\n  \"action\": $TOOL_NAME,\n  \"action_input\": $INPUT\n}\n```\n\nFollow this format:\n\nQuestion: input question to answer\nThought: consider previous and subsequent steps\nAction:\n```\n$JSON_BLOB\n```\nObservation: action result\n... (repeat Thought/Action/Observation N times)\nThought: I know what to respond\nAction:\n```\n{\n  \"action\": \"Final Answer\",\n  \"action_input\": \"Final response to human\"\n}\n\nBegin! Reminder to ALWAYS respond with a valid json blob of a single action. Use tools if necessary. Respond directly if appropriate. Format is Action:```$JSON_BLOB```then Observation"
                }
            ],
            "inferenceConfig": {
                "temperature": 0,
                "stopSequences": [
                    "\nObservation"
                ]
            },
            "toolConfig": {
                "tools": [
                    {
                        "toolSpec": {
                            "name": "VectorStore",
                            "description": "Useful for searching information from the knowledge base. This should be given the first priority while searching for information. It has information about Company1, Company2 and Company3's ESG reports in detail. If the information is not found, then the database tools must be used to find the answer",
                            "inputSchema": {
                                "json": {
                                    "type": "object",
                                    "properties": {
                                        "__arg1": {
                                            "title": "__arg1",
                                            "type": "string"
                                        }
                                    },
                                    "required": [
                                        "__arg1"
                                    ]
                                }
                            }
                        }
                    },
                    {
                        "toolSpec": {
                            "name": "sql_db_query",
                            "description": "Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.",
                            "inputSchema": {
                                "json": {
                                    "type": "object",
                                    "properties": {
                                        "query": {
                                            "type": "string",
                                            "description": "A detailed and correct SQL query."
                                        }
                                    },
                                    "required": [
                                        "query"
                                    ]
                                }
                            }
                        }
                    },
                    {
                        "toolSpec": {
                            "name": "sql_db_schema",
                            "description": "Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3",
                            "inputSchema": {
                                "json": {
                                    "type": "object",
                                    "properties": {
                                        "table_names": {
                                            "type": "string",
                                            "description": "A comma-separated list of the table names for which to return the schema. Example input: 'table1, table2, table3'"
                                        }
                                    },
                                    "required": [
                                        "table_names"
                                    ]
                                }
                            }
                        }
                    },
                    {
                        "toolSpec": {
                            "name": "sql_db_list_tables",
                            "description": "Input is an empty string, output is a comma-separated list of tables in the database.",
                            "inputSchema": {
                                "json": {
                                    "type": "object",
                                    "properties": {
                                        "tool_input": {
                                            "type": "string",
                                            "description": "An empty string",
                                            "default": ""
                                        }
                                    }
                                }
                            }
                        }
                    },
                    {
                        "toolSpec": {
                            "name": "sql_db_query_checker",
                            "description": "Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!",
                            "inputSchema": {
                                "json": {
                                    "type": "object",
                                    "properties": {
                                        "query": {
                                            "type": "string",
                                            "description": "A detailed and SQL query to be checked."
                                        }
                                    },
                                    "required": [
                                        "query"
                                    ]
                                }
                            }
                        }
                    }
                ]
            }
        },
        "inputTokenCount": 1760
    },
    "output": {
        "outputContentType": "application/json",
        "outputBodyJson": {
            "output": {
                "message": {
                    "role": "assistant",
                    "content": [
                        {
                            "text": "Certainly! I'll use the tools at my disposal to find the answer to your query about Company1's favorability rating on work-life balance and flexibility for the year 2022. Let's start by searching the knowledge base using the VectorStore tool.\n\nAction:\n```\n{\n  \"action\": \"VectorStore\",\n  \"action_input\": \"Company1 favorability rating work-life balance flexibility 2022\"\n}\n```"
                        }
                    ]
                }
            },
            "stopReason": "stop_sequence",
            "metrics": {
                "latencyMs": 3125
            },
            "usage": {
                "inputTokens": 1760,
                "outputTokens": 101,
                "totalTokens": 1861
            }
        },
        "outputTokenCount": 101
    }
}

后续调用（失败）日志 #

{
    "schemaType": "ModelInvocationLog",
    "schemaVersion": "1.0",
    "timestamp": "2025-02-07T08:12:04Z",
    "accountId": "821052193763",
    "identity": {
        "arn": "arn:aws:iam::821052193763:user/aws_demos"
    },
    "region": "us-east-1",
    "requestId": "fc1f2dfb-6e4f-46f4-af97-d122fd6afae1",
    "operation": "Converse",
    "modelId": "anthropic.claude-3-5-sonnet-20240620-v1:0",
    "errorCode": "ThrottlingException"
}

排查方向 #

• 并发请求限制：Bedrock除了token限额，还会限制每秒请求数（RPS）。检查账户在us-east-1区域针对该模型的RPS限额，后续调用间隔仅5秒，若存在并发触发逻辑，可能触发RPS限制。
• 操作类型限额差异：首次调用用的是ConverseStream，后续用的是Converse，两种操作的限额可能分开计算，确认Converse操作的单独限额。
• 账户实际限额：部分新账户或特殊账户的初始限额可能低于公开的400,000 tokens/分钟，通过AWS控制台Bedrock服务页面查看当前账户的具体生效限额。
• 区域资源瓶颈：us-east-1作为热门区域可能存在临时资源紧张，切换到其他支持Claude 3.5 Sonnet的区域（如us-west-2、eu-west-1）测试。
• 会话逻辑问题：检查后续调用的会话上下文处理逻辑，确保没有未释放的连接或重复请求的情况。

内容的提问来源于stack exchange，提问作者grammar