如何使用requests或内置库将Databricks DBFS文件上传至SharePoint

步骤1：读取DBFS中的文件内容 #

在Databricks环境中，有两种方式获取DBFS文件内容：

方式A：Notebook内直接读取 #

直接通过DBFS挂载的本地路径读取二进制内容：

file_path = "/dbfs/path/to/your/target-file.txt"
with open(file_path, "rb") as f:
    file_content = f.read()

方式B：通过Databricks REST API读取（外部环境适用） #

如果在Databricks集群外执行，调用DBFS的read接口获取内容（大文件需分片读取，示例为单次读取）：

import requests

databricks_token = "YOUR_DATABRICKS_PERSONAL_ACCESS_TOKEN"
databricks_instance = "https://<your-databricks-instance>.azuredatabricks.net"  # 替换为你的实例地址
dbfs_file_path = "/path/to/your/target-file.txt"

response = requests.get(
    f"{databricks_instance}/api/2.0/dbfs/read",
    headers={"Authorization": f"Bearer {databricks_token}"},
    params={"path": dbfs_file_path, "offset": 0, "length": 1048576}  # length按需调整，单位字节
)
response.raise_for_status()
file_content = response.content

步骤2：获取SharePoint访问令牌 #

通过Azure AD客户端凭证流获取令牌，需提前在Azure AD注册应用并授予SharePoint站点权限（如Sites.FullControl.All）：

import requests

tenant_id = "YOUR_AZURE_TENANT_ID"
client_id = "YOUR_REGISTERED_APP_CLIENT_ID"
client_secret = "YOUR_REGISTERED_APP_CLIENT_SECRET"
sharepoint_site_url = "https://<your-tenant>.sharepoint.com/sites/<your-target-site>"

token_url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/token"
payload = {
    "grant_type": "client_credentials",
    "client_id": client_id,
    "client_secret": client_secret,
    "resource": sharepoint_site_url.replace("https://", "")
}

token_response = requests.post(token_url, data=payload)
token_response.raise_for_status()
access_token = token_response.json()["access_token"]

步骤3：上传文件至SharePoint文档库 #

调用SharePoint REST API的文件添加接口，将二进制内容上传至指定文件夹：

# 目标路径：SharePoint服务器相对路径（可通过站点/_api/web/folders接口确认）
sharepoint_folder_path = "/sites/<your-target-site>/Shared Documents/Uploads"
file_name = "uploaded-file.txt"

upload_url = f"{sharepoint_site_url}/_api/web/getfolderbyserverrelativeurl('{sharepoint_folder_path}')/files/add(overwrite=true,url='{file_name}')"

headers = {
    "Authorization": f"Bearer {access_token}",
    "Accept": "application/json;odata=nometadata",
    "Content-Type": "application/octet-stream"
}

upload_response = requests.post(upload_url, headers=headers, data=file_content)
upload_response.raise_for_status()

print("文件上传成功" if upload_response.status_code == 200 else f"上传失败，状态码: {upload_response.status_code}")

关键注意事项 #

• 大文件处理：文件超过4MB时，需改用SharePoint的分片上传API（StartUpload/ContinueUpload/FinishUpload）。
• 权限验证：确保Azure AD应用拥有目标SharePoint站点的写入权限，且Databricks账号能读取指定DBFS文件。
• 路径准确性：SharePoint的服务器相对路径需与站点实际结构一致，避免路径错误导致请求失败。

内容的提问来源于stack exchange，提问作者Rana