使用HuggingFacePipeline调用flan-t5-base时遇ValueError问题求助

问题详情 #

我是生成式AI初学者，学习相关教程时遇到无法解决的问题，代码如下：

from langchain.llms import HuggingFacePipeline
from langchain import PromptTemplate, HuggingFaceHub, LLMChain
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline
import os
    
os.environ["HUGGINGFACEHUB_API_TOKEN"] = "my api" 
model_id = "google/flan-t5-base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)


pipeline = pipeline("text2text-generation", model=model, tokenizer=tokenizer, max_length=128)
local_llm = HuggingFacePipeline(pipeline=pipeline)

prompt = PromptTemplate(
    input_variables=["name"],
    template="Can you tell me about the politician {name}"
)

chain = LLMChain(llm=local_llm, prompt=prompt)
chain.run("Donald Trump")

运行后持续报错：

ValueError: The following model_kwargs are not used by the model: ['return_full_text'] (note: typos in the generate arguments will also show up in this list)

已在Jupyter Notebook、Google Colab、PyCharm中尝试，问题仍存在。

解决方法 #

问题根源是LangChain的HuggingFacePipeline默认会向模型传递return_full_text=True参数，但Flan-T5这类seq2seq模型并不支持该参数，需手动禁用。

有两种修改方式：

方式一：在HuggingFacePipeline中指定参数 #

修改创建local_llm的代码，添加model_kwargs配置：

local_llm = HuggingFacePipeline(pipeline=pipeline, model_kwargs={"return_full_text": False})

方式二：在transformers pipeline中直接设置 #

创建transformers pipeline时，直接加入return_full_text=False：

pipeline = pipeline(
    "text2text-generation",
    model=model,
    tokenizer=tokenizer,
    max_length=128,
    return_full_text=False
)

两种方式都能阻止LangChain传递模型不识别的参数，解决报错问题。

内容的提问来源于stack exchange，提问作者Arth1234