异步调用OpenAI接口：.py文件较Jupyter Notebook耗时过长原因咨询

为加速多轮OpenAI ChatCompletion API调用，我采用异步调用方式优化性能。该方案在Jupyter Notebook中平均耗时约3.5秒，但将相同代码迁移至.py文件运行时，平均耗时长达10秒。尝试使用concurrent.futures方案，结果仍类似。

运行代码 #

注：import nest_asyncio及nest_asyncio.apply()仅在Jupyter Notebook中需要，.py文件中可删除这两行。

import asyncio
import openai
import time
import nest_asyncio
nest_asyncio.apply()

openai.api_base = ""
openai.api_version = "2023-09-15-preview"
openai.api_key = ""

prompts = ["What are prime numbers", # some random questions
           "Translate this to Spanish : How are you", 
           "Explain the evolution of milkyway galaxy"]

async def process_prompt(prompt):
    loop = asyncio.get_event_loop()
    response = await loop.run_in_executor(None, lambda: openai.ChatCompletion.create(
        engine="development",
        messages=[{'role':'user','content':prompt}]
    ))
    return response.choices[0].message['content']

async def main():
    tasks = [process_prompt(prompt) for prompt in prompts]
    results = await asyncio.gather(*tasks)
    
    for result in results:
        print(result)

start=time.time()
asyncio.run(main())
end=time.time()
print('Time take',end-start)

问题原因分析 #

该差异并非直接由OpenAI限制导致，主要源于以下几点：

• 事件循环与线程池配置差异：Jupyter的事件循环经过特殊优化，nest_asyncio允许嵌套运行事件循环；而普通.py文件使用标准asyncio事件循环。另外，代码中loop.run_in_executor(None)默认使用concurrent.futures.ThreadPoolExecutor，Jupyter环境的线程池默认配置更宽松，标准Python环境默认线程池大小为CPU核心数*5，并发请求时会因线程排队增加总耗时。
• 环境初始化开销：.py文件每次运行都会重新初始化Python解释器、加载依赖库、建立API连接，这些开销在Jupyter中仅内核启动时发生一次，重复运行代码无需重复加载，因此Jupyter的耗时仅包含API请求时间，而.py文件包含初始化的额外耗时。
• 网络连接池复用差异：Jupyter默认维护持久HTTP连接池，减少了TCP握手、SSL协商的重复开销；而.py文件每次运行都需重新创建连接，增加了额外耗时。

优化建议 #

1. 显式配置线程池：指定更大的线程池大小，避免请求排队：

   from concurrent.futures import ThreadPoolExecutor
   
   # 初始化线程池
   executor = ThreadPoolExecutor(max_workers=10)
   
   async def process_prompt(prompt):
       loop = asyncio.get_event_loop()
       response = await loop.run_in_executor(executor, lambda: openai.ChatCompletion.create(
           engine="development",
           messages=[{'role':'user','content':prompt}]
       ))
       return response.choices[0].message['content']
   

2. 使用OpenAI原生异步SDK：直接调用异步客户端，无需用线程池包装同步方法，性能更优：

   import asyncio
   import time
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(
       api_base="你的api_base",
       api_key="你的api_key",
       api_version="2023-09-15-preview"
   )
   
   prompts = ["What are prime numbers",
              "Translate this to Spanish : How are you", 
              "Explain the evolution of milkyway galaxy"]
   
   async def process_prompt(prompt):
       response = await client.chat.completions.create(
           model="development",
           messages=[{"role": "user", "content": prompt}]
       )
       return response.choices[0].message.content
   
   async def main():
       tasks = [process_prompt(prompt) for prompt in prompts]
       results = await asyncio.gather(*tasks)
       
       for result in results:
           print(result)
   
   start=time.time()
   asyncio.run(main())
   end=time.time()
   print('Time take',end-start)
   

3. 复用连接池：在.py文件中显式配置HTTP连接池参数，确保SDK复用连接，减少网络握手开销。

内容的提问来源于stack exchange，提问作者learner123