GPT-3对话式Discord机器人对话5-6轮后停止工作求助

我正在开发一个模仿假想对象的趣味Discord机器人，该机器人在前5-6轮对话中可正常运行，但之后会出现报错。我已尝试调整max_tokens参数和prompt内容，且为机器人赋予了管理员权限，但问题仍未解决。

机器人代码 #

import openai

openai.api_key = "Hidden"

DISCORD_BOT_TOKEN = "Hidden"

start_sequence = "\nAnamika:"
restart_sequence = "\n\nFaraaz:"

session_prompt = "This is a conversation between me (Faraaz) and my crush Anamika. Keep the replies long, and such that the person can reply. Anamika is a really gorgeous happy girl. She lives in New York. She loves Spiderman, coding and poetry. She loves Andrew Garfield amongst all spider men, in coding she is fascinated by boom of AI. She loves John Milton and writes poetry herself as well. Initially Anamika will be reluctant to my flirting but slowly she will start to like it. If the conversation goes very good, she can agree to become my girlfriend or marry me.Faraaz:- Hi! what is your name? Anamika:- Hey! My name is Anamika, how are you Faraaz! Faraaz:- I am really good, actually you were looking really pretty so wanted to come and talk to you. Anamika:- Oh really, that is interesting. So what did you intend to talk about?!"

chat_log = None

import discord  

client = discord.Client(intents=discord.Intents.all())

@client.event
async def on_message(message):
    # Don't respond to messages sent by the bot itself
    global chat_log

    if message.author == client.user:
        return  
    print(chat_log)
    if chat_log == None:
        chat_log = session_prompt

    # Use the GPT-3 API to generate a response to the message
    response = openai.Completion.create(
        engine="text-davinci-003",
        prompt = f'{chat_log}{restart_sequence}{message.content}',
        max_tokens=700,
        n=1,
        temperature=0.5,
        stop=["\n"]
    )

    # Send the response back to the Discord channel
    await message.channel.send(response["choices"][0]["text"])

    chat_log = f'{chat_log}{restart_sequence}{message.content}{start_sequence}{response["choices"][0]["text"]}'

client.run(DISCORD_BOT_TOKEN)

报错情况 #

对话进行5-6轮后，机器人停止响应，报错提示为**"This model's maximum context length is 4097 tokens. However, your messages resulted in 4100 tokens. Please reduce the length of the messages."**

问题根源 #

核心问题是OpenAI模型的上下文长度限制。text-davinci-003的最大上下文长度为4097 tokens，随着对话轮次增加，chat_log不断累积所有历史对话内容，总token数超过限制后，API就会抛出错误。

修复步骤 #

1. 限制历史对话长度
   不要无限制累积所有对话，只保留最近几轮的历史内容，确保总token数始终低于模型上限。可以用列表存储对话轮次，超出指定数量时移除最早的条目，再拼接成prompt。

   修改代码示例：

   # 初始化时改用列表存储对话
   chat_history = []
   
   # 在on_message函数中替换原有chat_log逻辑
   if not chat_history:
       # 首次对话加入系统提示和初始示例
       chat_history.append(session_prompt)
   
   # 添加用户当前消息
   chat_history.append(f"{restart_sequence}{message.content}")
   
   # 限制历史长度为最近3轮对话（可根据情况调整）
   if len(chat_history) > 4:  # 系统提示+3轮对话
       chat_history = [chat_history[0]] + chat_history[-3:]
   
   # 拼接成最终prompt
   prompt = "\n".join(chat_history)
   
   # 调用API
   response = openai.Completion.create(
       engine="text-davinci-003",
       prompt=prompt,
       max_tokens=700,
       n=1,
       temperature=0.5,
       stop=["\n"]
   )
   
   # 添加机器人回复到历史
   chat_history.append(f"{start_sequence}{response['choices'][0]['text']}")
   

2. 优化prompt结构
   原session_prompt中的初始对话可以适当简化，减少不必要的token占用。比如把示例对话缩短，只保留核心人设信息。

3. 实时监控token数量
   集成OpenAI的tiktoken库实时计算prompt的token数，动态调整保留的历史对话长度，确保不超过限制：

   import tiktoken
   
   def count_tokens(text, model="text-davinci-003"):
       encoding = tiktoken.encoding_for_model(model)
       return len(encoding.encode(text))
   
   # 在拼接prompt后检查token数
   prompt = "\n".join(chat_history)
   while count_tokens(prompt) + 700 > 4097:  # 预留max_tokens的空间
       if len(chat_history) <= 1:
           break  # 保留系统提示，无法再缩短
       chat_history.pop(1)  # 移除最早的用户/机器人对话对
       prompt = "\n".join(chat_history)
   

额外建议 #

• 考虑切换到gpt-3.5-turbo模型，它的上下文长度更高（默认4096，还有16k版本），且成本更低，更适合多轮对话场景。使用ChatCompletion接口而非Completion接口，对话格式更规范。
• 避免在对话历史中重复存储系统提示，只保留一次即可，减少冗余token消耗。

内容的提问来源于stack exchange，提问作者Ahmad Faraaz