Azure搜索索引RAG向量搜索Chat Completions返回chunk_id异常问题

问题概述 #

使用包含文本拆分与嵌入生成的技能集创建索引后，通过Azure OpenAI聊天补全API获取结果及引用时，返回的chunk_id字段始终为0，无法为用户提供有意义的引用区分。而对原始文档进行关键词搜索时，能返回正确的chunk_id值。

原因推测 #

拆分技能会为每个文本块在索引中生成独立记录，但向量搜索返回结果时，未正确关联拆分后的块标识，导致所有引用的chunk_id均显示为0。

期望解决方式 #

• 让来自不同chunk的引用返回不同的chunk_id值
• 将拆分技能生成的数组索引与文档标题拼接，使标题变为类似doc1 Comms 2022.pdf - pt 1、doc1 Comms 2022.pdf - pt 2的格式，便于区分

目前发现的可行方案是通过Azure函数使用自定义Web技能，但该方案对于此类简单需求而言过于复杂，希望找到更简便的解决途径。

调用API的请求参数 #

{"data_sources": [{"type": "azure_search","parameters": {"endpoint": "https://xxxxx.search.windows.net","authentication":{"type":"api_key","key": "yyyyyyy"},"index_name": "xxxxxx","topNDocuments":5,"query_type": "vectorSimpleHybrid","vectorFilterMode": "preFilter","filter":"(search.ismatch('@{variables('region')}', 'region')) and (search.ismatch('@{variables('channel')}', 'channel'))","embeddingEndpoint":"xxxxx/openai/deployments/yyyyyyy/embeddings?api-version=2024-02-15-preview","embeddingKey":"xxxxxxx"}}],"messages": [{"role": "system","content": "You are..."},{"role": "user","content": "@{triggerBody()?['text']}"}],"temperature": 1.2,"top_p": 0.5,"frequency_penalty": 0,"presence_penalty": 0,"max_tokens": 2000,"stop": null}

返回示例对比 #

案例1：非分块文档的关键词搜索返回 #

"message": {"role": "assistant","content": "zzzzz [doc2].","end_turn": true,"context": {"citations": [{"content": "xxxxxxxxxx","title": "Process Hierarchy - 22-11-2018 11-01.pdf","url":"https://xxxxxxxxx.sharepoint.com/xxxxxxxxx/Process%20Hierarchy%20-%2022-11-2018%2011-01.pdf","filepath": "/xxxxxxxxx/Process Hierarchy - 22-11-2018 11-01.pdf","chunk_id": "31"}]}}

案例2：分块文档的向量搜索返回 #

{"id": "zzzzzzzzzzzzzz","model": "gpt-35-turbo","created": 1718180381,"object": "extensions.chat.completion","choices": [{"index": 0,"finish_reason": "stop","message": {"role": "assistant","content": "yyyyyyyyyyyyyyyyyyy","end_turn": true,"context": {"citations": [{"content": "xxxxxxxxxxxxxxx","title": "doc1 Comms 2022.pdf","url": "https://xxxxxxxxxx.blob.core.windows.net/xxxx/doc1%20%20Comms%202022.pdf","filepath": null,"chunk_id": "0"},{"content": "yyyyyyyyyyyyyyyy","title": "doc1 Comms 2022.pdf","url": "https://xxxxxxxxxx.blob.core.windows.net/xxxx/doc1%20%20Comms%202022.pdf","filepath": null,"chunk_id": "0"}]}}}

可见案例2中同一文档的不同chunk被返回两次，但chunk_id均为0，无法区分。

内容的提问来源于stack exchange，提问作者esteebie