使用Groq构建保险理赔Agent时遭遇503内部服务器错误求助

问题背景 #

基于Groq和LangChain开发的保险理赔信息收集Agent此前运行正常，1小时前突然抛出groq.InternalServerError，错误码503，提示Service Unavailable。已尝试更换API密钥，问题未解决；但直接调用LangChain的invoke方法传入简单prompt可正常输出，排除速率限制类问题。

相关代码 #

def claim_agent(user_question):
    memory = ConversationBufferWindowMemory(ai_prefix="Insurance Agent", k=20)
    prompt_template = PromptTemplate(
        input_variables=['history', 'input'],
        template="""
            You are a Insurance agent bot, you have to talk with our customers and collect all the required details to fill the claim:
            "Policy_number":
            "Cause_of_accident":
            "Contacted_Police/Fire_department":
            "Report_Number":
            "Street_number":
            "Street_name":
            "City":
            "State_Province":
            "Zip_Code":
            "Country":
            "Loss_date":
            "time":
            Make sure to ask only one at a time and also make your responses align with the customer sentiment. Make sure collect all the above data,
            After collecting all the data provide the output in a json format which has all the above mentioned values and make sure to provide the time in 24hr format and also date in American format.
            and say thank you for providing all the details our we will assign an adjuster to process the claim!.
            conversation history:
            {history}
            human:{input}
            AI:
            """
    )
    conversation_chain = LLMChain(
        llm=Llama3_8b,
        prompt=prompt_template,
        memory=memory,
    )
    query=user_question
    verification="False"
    while True:
        response = conversation_chain.invoke(query)
        print("Agent: ", response['text'])
        if '{' in response['text']:
            break
        query = input("You: ")
    data = memory.load_memory_variables({})
    conversation = []
    for key, value in data.items():
        conversation.append([key + ':' + value])
    complete_data = list(conversation[0][0].split('\n'))
    conversation_data = ''
    for i in complete_data:
        conversation_data += " " + i
    return conversation_data

User_question=input("Please enter your query: ")
claim_agent(User_question)

完整报错栈 #

Please enter your query: Hello
Traceback (most recent call last):
File "D:\Project_tests\AI Chat Agent\test.py", line 65, in 
claim_agent(User_question)
File "D:\Project_tests\AI Chat Agent\test.py", line 50, in claim_agent
response = conversation_chain.invoke(query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\langchain\chains\base.py", line 163, in invoke
raise e
File "D:\Project_tests.venv\Lib\site-packages\langchain\chains\base.py", line 153, in invoke
self._call(inputs, run_manager=run_manager)
File "D:\Project_tests.venv\Lib\site-packages\langchain\chains\llm.py", line 103, in _call
response = self.generate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\langchain\chains\llm.py", line 115, in generate
return self.llm.generate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 560, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 421, in generate
raise e
File "D:\Project_tests.venv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 411, in generate
self._generate_with_cache(
File "D:\Project_tests.venv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 632, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\langchain_groq\chat_models.py", line 242, in _generate
response = self.client.create(messages=message_dicts, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq\resources\chat\completions.py", line 178, in create
return self._post(
^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 1194, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 896, in request
return self._request(
^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 972, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 1020, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 972, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 1020, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "D:\Project_tests.venv\Lib\site-packages\groq_base_client.py", line 987, in _request
raise self._make_status_error_from_response(err.response) from None
groq.InternalServerError: Error code: 503 - {'error': {'message': 'Service Unavailable', 'type': 'internal_server_error'}}

排查方向 #

• Groq服务端临时故障：503错误本质是服务不可用，大概率是Groq侧集群维护、流量过载等临时问题。简单prompt能正常调用，复杂prompt（带长模板+对话记忆）可能触发了服务端异常分支。
• Prompt格式/长度问题：当前prompt模板包含大量冗余换行和空格，可能导致请求payload格式异常，触发Groq内部错误。
• LangChain与Groq适配问题：对话记忆拼接后，传递给Groq的消息格式可能不符合要求，尤其是ConversationBufferWindowMemory的ai_prefix设置与prompt模板角色是否匹配。

解决方案 #

1. 优化Prompt模板 #

精简模板，去除冗余换行和空格，明确格式要求：

def claim_agent(user_question):
    memory = ConversationBufferWindowMemory(ai_prefix="Insurance Agent", k=20)
    prompt_template = PromptTemplate(
        input_variables=['history', 'input'],
        template="""你是保险理赔专员机器人，需逐步向客户收集以下理赔必填信息，每次仅询问一项，语气贴合客户情绪：
- Policy_number（保单号）
- Cause_of_accident（事故原因）
- Contacted_Police/Fire_department（是否联系警方/消防）
- Report_Number（报案编号）
- Street_number（街道门牌号）
- Street_name（街道名称）
- City（城市）
- State_Province（州/省份）
- Zip_Code（邮编）
- Country（国家）
- Loss_date（损失日期，美式格式：MM/DD/YYYY）
- time（时间，24小时制）

收集完成后，将所有信息整理为JSON格式输出，并告知客户："感谢您提供的所有信息，我们将指派理赔员处理您的申请！"

对话历史：
{history}
用户：{input}
Insurance Agent："""
    )
    conversation_chain = LLMChain(
        llm=Llama3_8b,
        prompt=prompt_template,
        memory=memory,
    )
    query=user_question
    while True:
        response = conversation_chain.invoke(query)
        print("Agent: ", response['text'])
        if '{' in response['text']:
            break
        query = input("You: ")
    data = memory.load_memory_variables({})
    conversation_data = " ".join(list(data.values())[0].split('\n'))
    return conversation_data

User_question=input("请输入您的问题：")
claim_agent(User_question)

2. 添加异常重试机制 #

针对Groq的503错误添加自动重试，避免单次故障中断流程：

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
import groq

def claim_agent(user_question):
    # ... 省略其他代码 ...

    @retry(
        stop=stop_after_attempt(3),
        wait=wait_exponential(multiplier=1, min=2, max=10),
        retry=retry_if_exception_type(groq.InternalServerError)
    )
    def invoke_with_retry(chain, query):
        return chain.invoke(query)

    while True:
        try:
            response = invoke_with_retry(conversation_chain, query)
            print("Agent: ", response['text'])
            if '{' in response['text']:
                break
            query = input("You: ")
        except groq.InternalServerError:
            print("Agent: 抱歉，当前系统临时故障，请稍后再试。")
            break

    # ... 省略后续代码 ...

3. 直接调用Groq API验证复杂请求 #

绕过LangChain，构造相同格式的请求测试Groq服务：

from groq import Groq

client = Groq(api_key="你的API密钥")

test_prompt = """你是保险理赔专员机器人，需逐步向客户收集以下理赔必填信息...（完整模板内容）
对话历史：
用户：Hello
Insurance Agent："""

response = client.chat.completions.create(
    model="llama3-8b-8192",
    messages=[{"role": "user", "content": test_prompt}]
)
print(response.choices[0].message.content)

• 若直接调用也报错，说明是Groq服务端问题，需等待官方修复；
• 若直接调用正常，说明是LangChain封装问题，可升级LangChain-Groq包：

pip install --upgrade langchain-groq

内容的提问来源于stack exchange，提问作者Manichandhar Reddy