Cloud Scheduler触发Cloud Function返回500内部错误求助

配置信息 #

• 触发频率：0 9 1 * *
• 目标类型：HTTP
• 目标URL：https://europe-west2-project_name.cloudfunctions.net/xxxxx

已完成配置：

• 添加OIDC令牌
• 创建用于调用Cloud Function的服务账号

手动运行Cloud Scheduler失败，返回500内部错误，日志如下：

{
  "httpRequest": {
    "status": 500
  },
  "insertId": "132gtrbf1pw9rq",
  "jsonPayload": {
    "@type": "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished",
    "jobName": "projects/project_name/locations/europe-west2/jobs/cloud-scheduler-job",
    "status": "INTERNAL",
    "targetType": "HTTP",
    "url": "https://europe-west2-project_name.cloudfunctions.net/xxxx"
  },
  "logName": "projects/project_name/logs/cloudscheduler.googleapis.com%2Fexecutions",
  "receiveTimestamp": "2023-06-28T07:37:31.684339045Z",
  "resource": {
    "labels": {},
    "type": "cloud_scheduler_job"
  },
  "severity": "ERROR",
  "timestamp": "2023-06-28T07:37:31.684339045Z"
}

更新内容：审计日志 #

{
  "insertId": "-r6s7cscbc",
  "logName": "projects/project-name/logs/cloudaudit.googleapis.com%2Fdata_access",
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "authenticationInfo": {
      "principalEmail": "service-1234567890@gcf-admin-robot.iam.gserviceaccount.com"
    },
    "requestMetadata": {
      "requestAttributes": {
        "time": "2023-06-27T18:17:21.627074Z",
        "auth": {}
      },
      "destinationAttributes": {}
    },
    "serviceName": "cloudfunctions.googleapis.com",
    "methodName": "google.cloud.functions.v1.CloudFunctionsService.GetFunction",
    "authorizationInfo": [
      {
        "resource": "projects/project-name/locations/europe-west2/functions/cloud-function-name",
        "permission": "cloudfunctions.functions.get",
        "granted": true,
        "resourceAttributes": {}
      }
    ],
    "resourceName": "projects/project-name/locations/europe-west2/functions/cloud-function-name",
    "resourceLocation": {
      "currentLocations": [
        "europe-west2"
      ]
    }
  },
  "receiveTimestamp": "2023-06-27T18:17:21.927721083Z",
  "resource": {
    "labels": {
      "function_name": "cloud-function-name",
      "project_id": "project-name",
      "region": "europe-west2"
    },
    "type": "cloud_function"
  },
  "severity": "INFO",
  "timestamp": "2023-06-27T18:17:21.556303Z"
}

更新内容：Cloud Function Python代码 #

from google.cloud import storage
import zipfile
import datetime
import os

def move_files(data, context):
    source_bucket_name = 'source-input'
    destination_bucket_name = 'destination-input'
    client = storage.client()

    source_bucket = client.get_bucket(source_bucket_name)
    destination_bucket = client.get_bucket(destination_bucket_name)

    three_month_file = datetime.datetime.now() - datetime.timedelta(days=90)

    for blob in source_bucket.list_blobs(prefix='files/data/document'):
        blob_updated = blob.updated
        if blob_updated < three_month_file:
            #Create temporary local directory to store the zipped file
            temp_dir = 'temp'
            os.makedirs(temp_dir,exist_ok=True)

            #Store the file to the temporary local directory
            blob.store_filename(os.path.join(temp_dir,blob.name))
            #Zip the file
            zip_Name = os.path.join(temp_dir,f"{blob.name}.zip")
            with zipfile.ZipFile(zip_Name,'w') as zipfile_obj:
                zipfile_obj.write(os.path.join(temp_dir,blob.name),blob.name)

            #move the zipeed file to destination bucket
            destination_blob = destination_bucket.blob(f"{blob.name}.zip")
            destination_blob.upload_files(zip_Name)

            #Delete the original file and zipped file
            os.remove(os.path.join(temp_dir,blob.name))
            os.remove(zip_Name)

            #Delete the original from the source bucket
            blob.delete()
        #Remove the temporary local directory
        os.rmdir(temp_dir)

排查与修复方案 #

1. 代码错误修复 #

你的Python代码存在多个API调用错误和逻辑问题，直接导致Cloud Function执行失败返回500错误：

• 错误方法调用：blob.store_filename → 替换为blob.download_to_filename；destination_blob.upload_files → 替换为destination_blob.upload_from_filename
• 时区问题：datetime.datetime.now()改为datetime.datetime.utcnow()，避免本地时间与Cloud Function UTC时间偏差
• 目录清理错误：os.rmdir(temp_dir)放在循环内会重复执行，且无法删除非空目录，改用shutil.rmtree并移到finally块确保无论执行结果如何都清理
• 路径处理优化：避免使用完整blob路径作为本地文件名，改用os.path.basename(blob.name)防止目录结构冲突

修正后的代码：

from google.cloud import storage
import zipfile
import datetime
import os
import shutil

def move_files(data, context):
    source_bucket_name = 'source-input'
    destination_bucket_name = 'destination-input'
    client = storage.Client()

    source_bucket = client.get_bucket(source_bucket_name)
    destination_bucket = client.get_bucket(destination_bucket_name)

    # 使用UTC时间避免时区偏差
    three_month_ago = datetime.datetime.utcnow() - datetime.timedelta(days=90)
    temp_dir = 'temp'
    
    try:
        os.makedirs(temp_dir, exist_ok=True)
        
        for blob in source_bucket.list_blobs(prefix='files/data/document'):
            # 移除时区信息后比较时间
            blob_updated = blob.updated.replace(tzinfo=None)
            if blob_updated < three_month_ago:
                # 下载文件到临时目录，仅保留文件名
                local_file_name = os.path.basename(blob.name)
                local_file_path = os.path.join(temp_dir, local_file_name)
                blob.download_to_filename(local_file_path)
                
                # 压缩文件
                zip_file_path = os.path.join(temp_dir, f"{local_file_name}.zip")
                with zipfile.ZipFile(zip_file_path, 'w', zipfile.ZIP_DEFLATED) as zip_obj:
                    zip_obj.write(local_file_path, local_file_name)
                
                # 上传压缩文件到目标Bucket，可添加归档前缀分类
                dest_blob = destination_bucket.blob(f"archived/{local_file_name}.zip")
                dest_blob.upload_from_filename(zip_file_path)
                
                # 删除本地临时文件
                os.remove(local_file_path)
                os.remove(zip_file_path)
                
                # 删除源Bucket中的原文件
                blob.delete()
    finally:
        # 确保临时目录被清理
        if os.path.exists(temp_dir):
            shutil.rmtree(temp_dir)

2. 权限配置验证 #

从审计日志看，cloudfunctions.functions.get权限已授予，但需确保调用Cloud Function的服务账号拥有以下权限：

• storage.objects.get（读取源Bucket文件）
• storage.objects.create（写入目标Bucket）
• storage.objects.delete（删除源Bucket文件）
• cloudfunctions.functions.invoke（调用Cloud Function）

给服务账号绑定roles/storage.objectAdmin（针对源和目标Bucket）和roles/cloudfunctions.invoker（针对目标Cloud Function）角色即可。

3. Cloud Scheduler配置检查 #

• 确认OIDC令牌使用的服务账号已被授予roles/cloudfunctions.invoker角色，且绑定到目标Cloud Function
• 检查目标URL的项目名、函数名、区域完全匹配，无拼写错误
• 验证Cloud Function已成功部署，在控制台中手动测试函数是否能正常执行

内容的提问来源于stack exchange，提问作者Suba