在Firebase Functions调用Gemini-1.5-Pro预览版API处理视频时遇400错误

我在Firebase Cloud Functions中部署了Node.js函数，通过Vertex AI API调用gemini-1.5-pro-preview-0409模型处理GCP存储中的视频。代码直接复制自Vertex AI Studio，在AI Studio里测试视频和提示词都能正常运行，但调用云函数时日志一直报400 Bad Request错误。

我的代码 #

import {VertexAI} from '@google-cloud/vertexai';
const vertex_ai = new VertexAI({project: projectId, location: 'us-central1'});
const model = 'gemini-1.5-pro-preview-0409';

// Instantiate the models
const generativeModel = vertex_ai.preview.getGenerativeModel({
    model: model,
    generationConfig: {
        'maxOutputTokens': 8192,
        'temperature': 1,
        'topP': 0.95,
    },
    safetySettings: [
        {
            'category': 'HARM_CATEGORY_HATE_SPEECH',
            'threshold': 'BLOCK_MEDIUM_AND_ABOVE'
        },
        {
            'category': 'HARM_CATEGORY_DANGEROUS_CONTENT',
            'threshold': 'BLOCK_MEDIUM_AND_ABOVE'
        },
        {
            'category': 'HARM_CATEGORY_SEXUALLY_EXPLICIT',
            'threshold': 'BLOCK_MEDIUM_AND_ABOVE'
        },
        {
            'category': 'HARM_CATEGORY_HARASSMENT',
            'threshold': 'BLOCK_MEDIUM_AND_ABOVE'
        }
    ],
});

const text1 = {text: 'The video provided is a procedure with steps. Summarize the procedure.'};

export const outputSimFromVertexAI = onCall({timeoutSeconds: 900, memory: "1GiB"}, async (request) => {
    console.log('data is ')
    console.log(request.data)

    const videoPath = request.data.videoPath;

    console.log('video path is ', videoPath);
    const video1 = {
        fileData: {
            mimeType: 'video/mp4',
            fileUri: videoPath
        }
    };

    const req = {
        contents: [
            {role: 'user', parts: [video1, text1]}
        ]
    }

    console.log('video is ', video1)
    console.log('text1 is ', text1)

    const result = await generativeModel.generateContent(req);
    const aiResponse = JSON.stringify(await result.response);

    console.log('ai response is')
    console.log(aiResponse)

    return 'response generated successfully';
})

完整错误信息 #

Unhandled error ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Request contains an invalid argument.","status":"INVALID_ARGUMENT"}}
    at throwErrorIfNotOK (/workspace/node_modules/@google-cloud/vertexai/build/src/functions/post_fetch_processing.js:32:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async generateContent (/workspace/node_modules/@google-cloud/vertexai/build/src/functions/generate_content.js:51:5)
    at async file:///workspace/index.js:2142:20
    at async /workspace/node_modules/firebase-functions/lib/common/providers/https.js:467:26 {
  stackTrace: undefined
}

排查方向及解决方法 #

• 检查GCS视频文件权限：云函数默认使用的服务账号（你的项目ID@appspot.gserviceaccount.com）需要能读取目标GCS文件。给该账号添加Storage Object Viewer角色，或者直接在GCS桶的权限设置里赋予它storage.objects.get权限。
• 确认视频URI格式：确保传入的videoPath是标准的GCS格式，即gs://bucket-name/视频文件路径.mp4，不能用HTTP链接或者本地路径。
• 核对请求结构：对比Vertex AI Studio的请求格式，确认contents里的parts顺序和结构是否正确。比如fileData的mimeType要和视频实际类型完全匹配，不能写错。
• 检查云函数服务账号权限：确保云函数使用的服务账号拥有aiplatform.user角色，允许调用Vertex AI API。去IAM控制台给该账号添加这个角色。
• 验证视频是否符合模型限制：gemini-1.5-pro-preview-0409对视频的时长、文件大小有上限，检查你的视频是否超出了官方规定的限制。

内容的提问来源于stack exchange，提问作者user2646187