自定义DSPy LM Client调用ChainOfThought触发KeyError: 'temperature'的解决方法

问题场景 #

基于DSPy的LM类实现了自定义CustomLMClient类，代码如下：

from dsp import LM
import json

class CustomLMClient(LM):
    def __init__(self, model, **kwargs):
        self.model = model
        self.provider = "default"
        self.history = []
        self.llm = ChatOpenAI(model_name=self.model)
        self.kwargs = kwargs

    def basic_request(self, prompt: str, **kwargs):
        response = self.llm.invoke(prompt)
        response = response.json()

        self.history.append({
            "prompt": prompt,
            "response": response,
            "kwargs": kwargs,
        })
        return response

    def __call__(self,  prompt, only_completed=True, return_sorted=False, **kwargs):
        response = self.request(prompt, **kwargs)
        response = json.loads(response)
        completions = response["content"]        
        return [completions]

实例化并配置DSPy：

custom_lm_client = CustomLMClient(model = 'gpt-4-32k')
dspy.settings.configure(lm = custom_lm_client)

使用ChainOfThought模块时触发KeyError: 'temperature'：

# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).
qa = dspy.ChainOfThought('question -> answer')

# Run with the default LM configured with `dspy.configure` above.
response = qa(question="How many floors are in the castle David Gregory inherited?", temperature =0)
print(response.answer)

错误栈：

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[189], line 5
      2 qa = dspy.ChainOfThought('question -> answer')
      4 # Run with the default LM configured with `dspy.configure` above.
----> 5 response = qa(question="How many floors are in the castle David Gregory inherited?", temperature =0)
      6 print(response.answer)

File ~/Documents/Workspace/Autogen/pyautogen/lib/python3.12/site-packages/dspy/predict/predict.py:61, in Predict.__call__(self, **kwargs)
     60 def __call__(self, **kwargs):
---> 61     return self.forward(**kwargs)

File ~/Documents/Workspace/Autogen/pyautogen/lib/python3.12/site-packages/dspy/predict/chain_of_thought.py:59, in ChainOfThought.forward(self, **kwargs)
     57     signature = new_signature
     58     # template = dsp.Template(self.signature.instructions, **new_signature)
---> 59 return super().forward(signature=signature, **kwargs)

File ~/Documents/Workspace/Autogen/pyautogen/lib/python3.12/site-packages/dspy/predict/predict.py:76, in Predict.forward(self, **kwargs)
     74 # If temperature is 0.0 but its n > 1, set temperature to 0.7.
     75 temperature = config.get("temperature")
---> 76 temperature = lm.kwargs["temperature"] if temperature is None else temperature
     78 num_generations = config.get("n")
     79 if num_generations is None:

KeyError: 'temperature'

问题原因 #

DSPy的Predict模块会尝试从LM实例的kwargs字典中读取temperature值，但CustomLMClient初始化时如果没传入该参数，self.kwargs就不会包含这个键，从而触发KeyError。此外原代码还存在ChatOpenAI返回值解析错误的问题。

修改方案 #

1. 初始化时设置默认参数 #

在__init__方法中为kwargs添加默认的temperature值，确保self.kwargs始终包含该键，同时将默认参数传入ChatOpenAI实例。

2. 合并调用时的参数 #

在basic_request中合并实例默认参数和当前调用传入的参数，保证每次请求使用最新配置。

3. 修复返回值解析错误 #

ChatOpenAI.invoke()返回的是AIMessage对象，直接取content属性即可，无需调用.json()方法。

修正后的完整代码 #

from dsp import LM
from langchain.chat_models import ChatOpenAI

class CustomLMClient(LM):
    def __init__(self, model, **kwargs):
        self.model = model
        self.provider = "default"
        self.history = []
        # 设置默认参数，确保temperature始终存在
        default_kwargs = {"temperature": 0.7}
        default_kwargs.update(kwargs)
        self.kwargs = default_kwargs
        # 将默认参数传入ChatOpenAI实例
        self.llm = ChatOpenAI(model_name=self.model, **self.kwargs)

    def basic_request(self, prompt: str, **kwargs):
        # 合并实例默认参数与当前调用参数
        current_kwargs = {**self.kwargs, **kwargs}
        # 更新LLM实例的参数
        self.llm = ChatOpenAI(model_name=self.model, **current_kwargs)
        # 调用模型并获取内容
        response = self.llm.invoke(prompt)
        
        self.history.append({
            "prompt": prompt,
            "response": response.content,
            "kwargs": current_kwargs,
        })
        return response.content

    def __call__(self, prompt, only_completed=True, return_sorted=False, **kwargs):
        # 调用DSPy基类的request方法处理请求
        response = self.request(prompt, **kwargs)
        # 返回符合DSPy要求的列表格式
        return [response]

内容的提问来源于stack exchange，提问作者Rohan Prasad