如何使用Databricks部署Streamlit应用并实现Azure Blob文件上传

我希望通过Databricks App运行Streamlit应用。我已有可在Visual Studio Code中运行的Python代码（如下所示），该代码可启动Streamlit应用并将上传至Streamlit的文件保存到Azure Blob存储，但我从未创建过Databricks应用，请问如何在Databricks中部署该应用并实现相同的文件上传至Azure Blob存储的功能？

原本地运行代码 #

import streamlit as st
import pandas as pd
import time
import openpyxl
from azure.storage.blob import BlobServiceClient
import os

st.title("site-cluster mapping data")
df_gsl = pd.read_csv(r"C:\users\xxx.xxx\.venv\store_list.csv")
df_bar = df_gsl[['business_unit' , 'site_id']]
df_bar = df_bar.groupby('business_unit').site_id.count().reset_index()
st.bar_chart(data = df_bar, x = 'business_unit' , y = 'site_id')

# Replace with your actual values
connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxxxxxxxxxxxxxxxxxxxx;AccountKey=xxxxxxxxxxxxxxx;EndpointSuffix=core.windows.net"
container_name = "container-xxxxxxx"
#local_file_path = "sample_input.csv"
blob_name = "sample_input.csv"

def upload_blob(connection_string, container_name, local_file_path , blob_name , df):
    # Create a BlobServiceClient
    blob_service_client = BlobServiceClient.from_connection_string(connection_string)

    # Create a container if it doesn't exist
    container_client = blob_service_client.get_container_client(container_name)
    
    # Upload the CSV file
    try:
        container_client.upload_blob(name = 'demo_input.csv', data = df.to_csv(), overwrite=True)
        st.write("file has been succesfully uploaded to the blob storage")
    except Exception as e:
        st.error("error uploading the file, please contact code owner")

部署步骤与代码调整 #

一、准备Databricks环境与依赖 #

• 在Databricks工作区创建新的笔记本，或通过左侧菜单栏「Apps」入口创建Databricks App。
• 安装所需依赖包，在笔记本中运行：

  %pip install streamlit azure-storage-blob pandas openpyxl
  

二、代码适配Databricks环境 #

原代码需做两处关键调整：

1. 本地文件路径替换：把原代码中本地store_list.csv的路径，替换为Databricks可访问的路径——可以是上传到工作区的路径（格式如/Workspace/你的文件夹路径/store_list.csv），或是Azure Blob存储的SAS URL/ADLS路径。
2. 敏感信息安全处理：不要硬编码Azure Blob的连接字符串，改用Databricks Secrets管理：
   • 在Databricks工作区创建Secret Scope，存储connection_string和container_name
   • 代码中通过dbutils.secrets.get()获取敏感信息

调整后的示例代码：

import streamlit as st
import pandas as pd
from azure.storage.blob import BlobServiceClient
import dbutils

# 从Databricks Secrets读取敏感信息
connection_string = dbutils.secrets.get(scope="azure_blob_scope", key="connection_string")
container_name = dbutils.secrets.get(scope="azure_blob_scope", key="container_name")

st.title("site-cluster mapping data")
# 替换为Databricks工作区路径或Blob存储路径
df_gsl = pd.read_csv("/Workspace/你的文件夹路径/store_list.csv")
df_bar = df_gsl[['business_unit' , 'site_id']]
df_bar = df_bar.groupby('business_unit').site_id.count().reset_index()
st.bar_chart(data=df_bar, x='business_unit', y='site_id')

def upload_blob(connection_string, container_name, df):
    blob_service_client = BlobServiceClient.from_connection_string(connection_string)
    container_client = blob_service_client.get_container_client(container_name)
    try:
        container_client.upload_blob(name='demo_input.csv', data=df.to_csv(index=False), overwrite=True)
        st.success("文件已成功上传至Blob存储")
    except Exception as e:
        st.error(f"上传失败：{str(e)}")

# 补充Streamlit文件上传交互逻辑
uploaded_file = st.file_uploader("上传CSV文件", type=["csv"])
if uploaded_file is not None:
    df_upload = pd.read_csv(uploaded_file)
    st.dataframe(df_upload)
    if st.button("上传至Blob存储"):
        upload_blob(connection_string, container_name, df_upload)

三、部署为Databricks App #

1. 点击Databricks左侧菜单栏「Apps」→「Create app」
2. 选择「Streamlit」作为应用类型，填写应用名称，选择关联集群（或新建集群）
3. 将调整后的代码粘贴到应用代码编辑器中
4. 点击「Deploy」完成部署，等待生成应用访问URL

四、功能验证 #

• 访问部署后的应用URL，确认图表正常显示
• 上传测试CSV文件，点击上传按钮，检查Azure Blob存储中是否生成demo_input.csv文件

内容的提问来源于stack exchange，提问作者Ian