如何用PyDrive追加更新Google Drive的CSV文件,无需重新连接Data Studio
问题描述
我需要更新Google Drive上的CSV文件(该文件关联Google Data Studio仪表盘),此前用以下代码实现:
previous_GDS_df = pd.read_excel(path_to_GDS_file) pd.concat(objs=[previous_GDS_df, df_GDS]).to_excel(path_to_GDS_file, index=False) f = drive.CreateFile({'id': spreadsheet_id}) f.SetContentFile(path_to_GDS_file) f.Upload()
参数说明:
previous_GDS_df:待更新的CSV文件内容path_to_GDS_file:本地CSV文件路径,用于修改操作df_GDS:需追加到Drive文件的新数据DataFrame
我的思路是提取原有内容、追加新内容后,通过SetContentFile覆盖Drive文件上传,但每次更新后必须重新连接Data Studio仪表盘——推测是因为SetContentFile会完全删除重写原有文件,导致文件的底层标识变化。
我尝试了Drive API的update方法,仍会重写整个文件,还是需要重新连接Data Studio:
from __future__ import print_function import os.path from google.auth.transport.requests import Request from google.oauth2.credentials import Credentials from google_auth_oauthlib.flow import InstalledAppFlow from googleapiclient.discovery import build from googleapiclient.errors import HttpError from apiclient.http import MediaFileUpload SCOPES = ['https://www.googleapis.com/auth/drive'] # If modifying these scopes, delete the file token.json. def getCreds(): # Authentication creds = None # The file token.json stores the user's access and refresh tokens, and is # created automatically when the authorization flow completes for the first # time. if os.path.exists('token.json'): creds = Credentials.from_authorized_user_file('token.json', SCOPES) # If there are no (valid) credentials available, let the user log in. if not creds or not creds.valid: if creds and creds.expired and creds.refresh_token: creds.refresh(Request()) else: flow = InstalledAppFlow.from_client_secrets_file( 'credentials.json', SCOPES) creds = flow.run_local_server(port=0) # Save the credentials for the next run with open('token.json', 'w') as token: token.write(creds.to_json()) return creds def updateFile(service, spreadsheet_id, path_to_GDS_file): # Call the API media = MediaFileUpload(path_to_GDS_file, mimetype='application/vnd.google-apps.spreadsheet', resumable=True) res = service.files().update(fileId=spreadsheet_id,media_body=media,fields="*").execute() return res def main(spreadsheet_id, path_to_GDS_file): creds = getCreds() service = build('drive', 'v3', credentials=creds) updateFile(service, spreadsheet_id, path_to_GDS_file) if __name__ == '__main__': main()
请问如何实现直接向Google Drive上的CSV文件追加行,无需重新连接Data Studio?
解决方案
核心逻辑是不重写整个文件,直接向文件内容末尾追加数据行,这样文件的ID和底层元数据保持不变,Data Studio的连接会自动识别更新,无需重新配置。
推荐使用Google Sheets API实现(即使原文件是CSV格式,Drive会自动将其转换为Sheets格式供API操作,操作后仍可保持CSV格式,不影响Data Studio读取),具体步骤如下:
1. 准备工作
安装依赖库:
pip install pandas google-api-python-client google-auth-httplib2 google-auth-oauthlib
更新权限范围,同时包含Drive和Sheets API的访问权限:
SCOPES = ['https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive']
2. 代码实现
以下代码直接操作Drive上的文件,无需下载本地修改后再上传:
import os import pandas as pd from google.auth.transport.requests import Request from google.oauth2.credentials import Credentials from google_auth_oauthlib.flow import InstalledAppFlow from googleapiclient.discovery import build SCOPES = ['https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive'] def get_creds(): creds = None # 读取已保存的凭证 if os.path.exists('token.json'): creds = Credentials.from_authorized_user_file('token.json', SCOPES) # 凭证无效或过期时刷新/重新授权 if not creds or not creds.valid: if creds and creds.expired and creds.refresh_token: creds.refresh(Request()) else: flow = InstalledAppFlow.from_client_secrets_file('credentials.json', SCOPES) creds = flow.run_local_server(port=0) # 保存凭证供下次使用 with open('token.json', 'w') as token: token.write(creds.to_json()) return creds def append_rows(spreadsheet_id, new_data_df): creds = get_creds() # 初始化Sheets API服务 service = build('sheets', 'v4', credentials=creds) # 将DataFrame转换为API可接受的列表格式 data_rows = new_data_df.values.tolist() # 若首次追加需要写入表头,取消下面注释(后续追加请勿重复执行,避免表头重复) # data_rows.insert(0, new_data_df.columns.tolist()) # 调用API追加行 body = {'values': data_rows} response = service.spreadsheets().values().append( spreadsheetId=spreadsheet_id, range='A1', # 自动定位到工作表最后一行的下一行追加 valueInputOption='RAW', # 按原始格式写入,避免自动格式转换 body=body ).execute() print(f"成功追加 {response['updates']['updatedRows']} 行数据") if __name__ == '__main__': # 替换为你的目标文件ID和新数据DataFrame TARGET_FILE_ID = '你的Google Drive文件ID' sample_new_data = pd.DataFrame({ '订单ID': ['ORD-001', 'ORD-002'], '金额': [120.5, 89.0], '日期': ['2024-05-01', '2024-05-02'] }) append_rows(TARGET_FILE_ID, sample_new_data)
3. 关键注意事项
- 该方法仅修改文件内容,不改变文件本身的ID和元数据,因此Data Studio仪表盘无需重新连接,会自动同步更新后的数据。
- 若原文件是纯CSV,Drive会在后台转换为Sheets格式供API操作,操作完成后仍可在Drive中以CSV格式查看或下载,不影响Data Studio的数据源配置。
valueInputOption='RAW'确保数据按原始格式写入,避免API自动转换日期、数字格式导致的匹配问题。
内容的提问来源于stack exchange,提问作者PaulFaguet
相关产品推荐
相关产品推荐

