基于Haystack的RAG流水线编译正常但返回空响应问题排查

我基于Haystack搭建的RAG流水线能正常编译运行，但始终返回空响应。最初怀疑是嵌入模型与LLM不兼容，于是将嵌入模型替换为和LLM同基于Mistral的版本，但问题依旧。加载嵌入器时还出现提示：Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

代码细节 #

模型配置 #

mymodel = "occiglot/occiglot-7b-eu5-instruct"  # llm模型
# embedding_model = "Alibaba-NLP/gte-Qwen2-7B-instruct"  # 旧嵌入模型
embedding_model = "intfloat/e5-mistral-7b-instruct"

导入代码 #

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.generators import HuggingFaceLocalGenerator

嵌入器与生成器初始化 #

embedder = SentenceTransformersDocumentEmbedder(model=embedding_model)
text_embedder = SentenceTransformersTextEmbedder(model=embedding_model)
generator = HuggingFaceLocalGenerator(model=mymodel)

索引流水线 #

indexing_pipeline = Pipeline()

indexing_pipeline.add_component("converter", MarkdownToDocument())
indexing_pipeline.add_component("splitter", DocumentSplitter(split_by="sentence", split_length=2))
indexing_pipeline.add_component("embedder", embedder)
indexing_pipeline.add_component("writer", DocumentWriter(document_store))

indexing_pipeline.connect("converter.documents", "splitter.documents")
indexing_pipeline.connect("splitter.documents", "embedder.documents")
indexing_pipeline.connect("embedder", "writer")

查询流水线 #

query_pipeline = Pipeline()

query_pipeline.add_component("text_embedder", text_embedder)
query_pipeline.add_component("retriever", MilvusEmbeddingRetriever(document_store=document_store, top_k=3))
query_pipeline.add_component("prompt_builder", PromptBuilder(template=prompt_template))
query_pipeline.add_component("generator", generator)

query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")
query_pipeline.connect("retriever.documents", "prompt_builder.documents")
query_pipeline.connect("prompt_builder", "generator") 

运行代码 #

indexing_pipeline.run({
    "converter": {"sources": [file_path]},
})
results = query_pipeline.run({
    "text_embedder": {"text": question},
})
print("RAG answer:", results["generator"]["replies"][0])

已尝试的修改 #

1. 将流水线拆分为独立的索引流水线和查询流水线
2. 使用如下提示词模板：

prompt_template = """Answer the following query based on the provided context. If the context does
                     not include an answer, reply with 'I don't know'.\n
                     Query: {{query}}
                     Documents:
                     {% for doc in documents %}
                        {{ doc.content }}
                     {% endfor %}
                     Answer:
                  """

3. 测试了Alibaba-NLP/gte-large-en-v1.5嵌入模型

当前状态 #

修改后仍返回空响应，query_pipeline.run()的完整输出为：{'generator': {'replies': ['\n']}}

排查建议 #

• 检查检索结果有效性：在查询流水线运行后，打印results["retriever"]["documents"]，确认是否从Milvus中检索到了实际的文档内容。如果检索结果为空，说明索引环节可能存在文档未写入、嵌入向量不匹配等问题。
• 修复提示词变量传递：当前提示词模板中的{{query}}没有被正确赋值——查询流水线仅传递了嵌入向量到检索器，未将用户的question文本传给prompt_builder。需要调整运行代码：

  results = query_pipeline.run({
      "text_embedder": {"text": question},
      "prompt_builder": {"query": question}
  })
  

• 调整LLM生成参数：给HuggingFaceLocalGenerator添加生成参数，避免默认配置导致空输出：

  generator = HuggingFaceLocalGenerator(
      model=mymodel,
      generation_kwargs={"max_new_tokens": 200, "temperature": 0.1}
  )
  

• 验证嵌入模型兼容性：暂时替换为轻量通用嵌入模型（如all-MiniLM-L6-v2），排除特殊token提示带来的嵌入效果问题。
• 检查索引环节的文档处理：确认索引流水线运行后，文档是否被正确拆分、嵌入并写入Milvus。可以直接查询Milvus中的向量数量，或打印嵌入后的文档内容验证流程正确性。

内容的提问来源于stack exchange，提问作者ArieAI