如何通过llama_index获取OpenAI响应中的finish_reason？

要从LlamaIndex的查询响应中提取OpenAI返回的finish_reason字段，可通过以下两种方式实现：

方法一：自定义LLM回调捕获原始响应 #

利用LlamaIndex的回调系统，在LLM调用结束时捕获OpenAI的完整原始响应，从中解析finish_reason：

from llama_index.core.callbacks.base_handler import BaseCallbackHandler
from llama_index.core.callbacks import CallbackManager
from llama_index.llms.openai import OpenAI

class FinishReasonCallback(BaseCallbackHandler):
    def __init__(self):
        self.finish_reason = None

    def on_llm_end(self, response, **kwargs):
        # 解析OpenAI的原始响应对象
        raw_response = response.raw
        if raw_response and hasattr(raw_response, 'choices'):
            self.finish_reason = raw_response.choices[0].finish_reason

# 实例化自定义回调并绑定到回调管理器
finish_reason_callback = FinishReasonCallback()
callback_manager = CallbackManager([finish_reason_callback])

# 初始化带回调的OpenAI LLM实例
llm = OpenAI(callback_manager=callback_manager)

# 构建查询引擎时传入自定义LLM
query_engine = index.as_query_engine(llm=llm)

# 执行查询
response = query_engine.query("what is this document about?")

# 获取并打印finish_reason
print(f"Finish Reason: {finish_reason_callback.finish_reason}")

方法二：手动调用OpenAI API（结合LlamaIndex提示） #

如果需要对OpenAI调用做更灵活的参数定制，可以通过LlamaIndex生成提示后，直接调用OpenAI API解析响应：

from llama_index.core import PromptTemplate
from openai import OpenAI as OpenAIClient

# 获取查询引擎使用的提示模板（以文本问答模板为例）
prompt_template = query_engine.get_prompts()["response_synthesizer:text_qa_template"]

# 生成填充后的查询提示
prompt = prompt_template.format(
    question="what is this document about?",
    context_str="你的文档上下文内容"
)

# 直接调用OpenAI API
client = OpenAIClient()
openai_response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": prompt}]
)

# 提取finish_reason字段
finish_reason = openai_response.choices[0].finish_reason
print(f"Finish Reason: {finish_reason}")

说明 #

• 方法一适合在LlamaIndex原有工作流中无缝集成，自动捕获所有LLM调用的finish_reason
• 方法二更灵活，支持自定义OpenAI API的额外参数（如温度、最大令牌数等）

内容的提问来源于stack exchange，提问作者Ire00