Langchain SQL Agent连接Databricks Unity Catalog表报错：资源未找到

我尝试用Langchain SQL Agent连接Databricks Unity Catalog中的表，执行agent_executor.run("Question asked?")时触发错误：InvalidRequestError: Resource not found。目标表training_data位于dslakehouse.default下，相关代码如下：

%pip install langchain==0.0.279 databricks-sql-connector openai==0.28.0

from langchain.agents import create_sql_agent
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.sql_database import SQLDatabase
from langchain import OpenAI
from langchain.chat_models import ChatOpenAI
import os
from langchain.agents.agent_types import AgentType
from langchain.chat_models import AzureChatOpenAI


db = SQLDatabase.from_databricks(catalog="dslakehouse", schema="default")
api_key = "XXX" 


MY_ENDPOINT = "https://XXX.azure.com/"
MY_DEPLOYMENT_NAME = "XXX"

llm = AzureChatOpenAI(
                  deployment_name=MY_DEPLOYMENT_NAME,
                  model_name='gpt-4o',
                  model_version='2024-05-13',
                  temperature = 0,
                  openai_api_key= api_key,
                  openai_api_base=MY_ENDPOINT,
                  openai_api_type='azure',
                  openai_api_version='2024-05-13',
                  max_retries=4,
)

toolkit = SQLDatabaseToolkit(db=db, llm=llm)

agent_executor = create_sql_agent(
    llm=llm,
    toolkit=toolkit,
    verbose=True,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    engine="openai-ds-test-4o", 
)

排查与解决方法 #

• Databricks连接参数缺失
  SQLDatabase.from_databricks仅指定catalog和schema不足以建立连接，必须补充Databricks工作区的核心连接参数：主机地址、集群HTTP路径、个人访问令牌（PAT）。修正后的连接代码：

  db = SQLDatabase.from_databricks(
      catalog="dslakehouse",
      schema="default",
      host="your-databricks-workspace-host",  # 示例：adb-xxxxxx.azuredatabricks.net
      http_path="your-cluster-http-path",      # 示例：sql/protocolv1/o/xxxxxx/xxxxxx
      token="your-databricks-pat"              # 从Databricks用户设置生成的个人访问令牌
  )
  

  这些参数可在Databricks工作区的集群详情页（JDBC/ODBC选项卡）和用户设置中获取。

• 权限不足
  确保你的PAT对应的账号拥有目标资源的访问权限：

  • 对catalog dslakehouse的USAGE权限
  • 对schema dslakehouse.default的USAGE权限
  • 对表dslakehouse.default.training_data的SELECT权限
    可在Databricks SQL中执行以下语句补全权限：

  GRANT USAGE ON CATALOG dslakehouse TO `your-username`;
  GRANT USAGE ON SCHEMA dslakehouse.default TO `your-username`;
  GRANT SELECT ON TABLE dslakehouse.default.training_data TO `your-username`;
  

• 移除无效Agent参数
  create_sql_agent方法中不存在engine参数，需删除该参数，修正后的Agent初始化代码：

  agent_executor = create_sql_agent(
      llm=llm,
      toolkit=toolkit,
      verbose=True,
      agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION
  )
  

• 升级依赖版本
  你使用的Langchain 0.0.279版本较旧，可能存在Unity Catalog连接的兼容性问题，建议升级到稳定新版本：

  %pip install langchain>=0.1.0 databricks-sql-connector>=3.0.0 openai>=1.0.0
  

内容的提问来源于stack exchange，提问作者Mikee