如何在Databricks中下载SharePoint文件？认证下载遇阻求助

方法1：Azure AD服务主体认证（自动化场景首选） #

适合无人值守的定时任务或流水线，通过服务主体完成非交互式认证。

前置步骤 #

1. 在Azure AD中注册一个应用程序，记录tenant_id、client_id、client_secret。
2. 给该应用授予SharePoint的应用权限（如Sites.Read.All，或针对特定站点的权限），并完成管理员同意（必须，否则权限不生效）。
3. 在Databricks中创建Secret Scope，将上述凭据存储进去，避免硬编码。

代码实现 #

先安装依赖（若集群未预装）：

%pip install requests msal

下载文件的Python代码：

import requests
import msal

# 从Databricks Secrets读取凭据
tenant_id = dbutils.secrets.get("your-secret-scope", "sp-tenant-id")
client_id = dbutils.secrets.get("your-secret-scope", "sp-client-id")
client_secret = dbutils.secrets.get("your-secret-scope", "sp-client-secret")

# 配置SharePoint信息
site_url = "https://<your-tenant>.sharepoint.com/sites/<your-site-name>"
file_relative_path = "/sites/<your-site-name>/<document-library>/<folder-path>/<target-file>"
download_target = "/dbfs/mnt/<your-mount-point>/<saved-file-name>"  # 保存到DBFS或本地路径

# 获取访问令牌
authority = f"https://login.microsoftonline.com/{tenant_id}"
scope = [f"{site_url}/.default"]
app = msal.ConfidentialClientApplication(client_id, authority=authority, client_credential=client_secret)
token_result = app.acquire_token_for_client(scopes=scope)

if "access_token" in token_result:
    headers = {
        "Authorization": f"Bearer {token_result['access_token']}",
        "Accept": "application/json;odata=verbose"
    }
    # 调用SharePoint REST API下载文件
    file_api_endpoint = f"{site_url}/_api/web/getfilebyserverrelativeurl('{file_relative_path}')/$value"
    response = requests.get(file_api_endpoint, headers=headers)
    response.raise_for_status()  # 抛出HTTP错误
    
    # 写入文件
    with open(download_target, "wb") as f:
        f.write(response.content)
    print(f"文件已成功下载至: {download_target}")
else:
    print(f"令牌获取失败: {token_result.get('error_description')}")

方法2：用户交互式认证（手动运行场景） #

适合临时手动执行的任务，通过设备码完成用户身份认证。

代码实现 #

同样先安装依赖：

%pip install requests msal

下载代码：

import requests
import msal

# 读取配置（client_id和tenant_id可存Secrets）
tenant_id = dbutils.secrets.get("your-secret-scope", "tenant-id")
client_id = dbutils.secrets.get("your-secret-scope", "client-id")
site_url = "https://<your-tenant>.sharepoint.com/sites/<your-site-name>"
file_relative_path = "/sites/<your-site-name>/<document-library>/<folder-path>/<target-file>"
download_target = "/dbfs/mnt/<your-mount-point>/<saved-file-name>"

# 初始化设备流认证
authority = f"https://login.microsoftonline.com/{tenant_id}"
scope = [f"{site_url}/Sites.Read.All"]
app = msal.PublicClientApplication(client_id, authority=authority)
device_flow = app.initiate_device_flow(scopes=scope)

if "user_code" not in device_flow:
    raise ValueError(f"设备流初始化失败: {device_flow.get('error_description')}")

# 提示用户完成认证
print(device_flow["message"])
token_result = app.acquire_token_by_device_flow(device_flow)

if "access_token" in token_result:
    headers = {
        "Authorization": f"Bearer {token_result['access_token']}",
        "Accept": "application/json;odata=verbose"
    }
    file_api_endpoint = f"{site_url}/_api/web/getfilebyserverrelativeurl('{file_relative_path}')/$value"
    response = requests.get(file_api_endpoint, headers=headers)
    response.raise_for_status()
    
    with open(download_target, "wb") as f:
        f.write(response.content)
    print(f"文件已成功下载至: {download_target}")
else:
    print(f"认证失败: {token_result.get('error_description')}")

常见问题排查 #

• 权限错误：确保授予的是应用权限而非委派权限，且已完成管理员同意；若仅需访问特定站点，可配置更精细的站点权限而非全局权限。
• 路径错误：file_relative_path必须是SharePoint的服务器相对路径，可通过站点文档库的"查看属性"获取正确路径。
• 凭据暴露：绝对禁止在代码中硬编码client_secret等敏感信息，必须使用Databricks Secrets管理。
• 网络连通性：确认Databricks集群可访问SharePoint API端点（https://<your-tenant>.sharepoint.com），若集群在虚拟网络中，需确保出站规则允许该地址。

内容的提问来源于stack exchange，提问作者Dimitar Grigorov