配置Groq模型后LlamaIndex仍调用OpenAI API的问题排查

尝试构建数据库检索Agent，已通过Settings指定Groq的llama-3.3-70b-versatile模型及Groq API密钥，但执行查询时LlamaIndex仍尝试调用OpenAI API，抛出APIConnectionError错误。

相关代码：

model = 'llama-3.3-70b-versatile'
model_hf_emb = 'BAAI/bge-m3'
Settings.llm = Groq(model=model, groq_api_key=key)
Settings.embed_model = HuggingFaceEmbedding(model_name=model_hf_emb)

sql_database = SQLDatabase(engine)
table_node_map = SQLTableNodeMapping(sql_database)

table_schema_obj = []
for table_name in metadata_obj.tables.keys():
    table_schema_obj.append(
        SQLTableSchema(table_name=table_name))
object_index = ObjectIndex.from_objects(table_schema_obj, table_node_map, VectorStoreIndex)
obj_retriever = object_index.as_retriever(similarity_top_k=1)
query_engine = SQLTableRetrieverQueryEngine(
    sql_database=sql_database,
    table_retriever=obj_retriever
)

resp = query_engine.query('Quais são os clientes cadastrados?')

报错信息：

WARNING:llama_index.llms.openai.utils:Retrying llama_index.llms.openai.base.OpenAI._chat in 1.0 seconds as it raised APIConnectionError: Connection error..
WARNING:llama_index.llms.openai.utils:Retrying llama_index.llms.openai.base.OpenAI._chat in 1.4790521048410215 seconds as it raised APIConnectionError: Connection error..
---------------------------------------------------------------------------
LocalProtocolError                        Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/httpx/_transports/default.py in map_httpcore_exceptions()
    100     try:
--> 101         yield
    102     except Exception as exc:

50 frames
LocalProtocolError: Illegal header value b'Bearer '

The above exception was the direct cause of the following exception:

LocalProtocolError                        Traceback (most recent call last)
LocalProtocolError: Illegal header value b'Bearer '

The above exception was the direct cause of the following exception:

APIConnectionError                        Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/openai/_base_client.py in _request(self, cast_to, options, retries_taken, stream, stream_cls)
   1028 
   1029             log.debug("Raising connection error")
--> 1030             raise APIConnectionError(request=request) from err
   1031 
   1032         log.debug(

APIConnectionError: Connection error.

核心原因 #

SQLTableRetrieverQueryEngine不会自动继承全局Settings中配置的LLM，若初始化时未显式传入llm参数，组件会默认使用OpenAI的LLM。由于未配置OpenAI的API密钥，请求时生成非法的Authorization头（空Bearer令牌），进而触发连接错误。

解决步骤 #

1. 显式传入LLM参数
   修改SQLTableRetrieverQueryEngine的初始化代码，将全局配置的Groq LLM传入：

   query_engine = SQLTableRetrieverQueryEngine(
       sql_database=sql_database,
       table_retriever=obj_retriever,
       llm=Settings.llm  # 显式指定使用Groq模型
   )
   

2. 验证API密钥有效性
   确认key变量已正确加载Groq的API密钥，避免后续Groq调用出现权限问题。

内容的提问来源于stack exchange，提问作者Sergio Pantano