Firebase定时函数本地正常，生产环境请求返回503错误求助

问题描述 #

我编写了一个Firebase定时函数用于抓取指定URL的RSS内容，代码如下：

import * as admin from "firebase-admin";
import {onSchedule} from "firebase-functions/v2/scheduler";
import axios from 'axios';

export const getKittens = onSchedule("0 * * * *", async (event) => {
    const responseBody = (await axios("https://news.google.com/rss/search?hl=he&gl=IL&ceid=IL%3Ahe&oc=11&q=kittens")).data;
    admin.firestore().collection("news").doc("latest").set({response: responseBody});
});

该函数在本地模拟器中可正常获取RSS响应，但部署到生产环境后请求无返回值且触发503错误。我已尝试使用Node原生https、node-fetch及axios，并添加了User-Agent请求头，问题仍未解决。

相关日志片段 #

ERROR 2023-10-19T21:00:14.341994Z AxiosError: Request failed with status code 503 at settle (/workspace/node_modules/axios/dist/node/axios.cjs:1913:12) at IncomingMessage.handleStreamEnd (/workspace/node_modules/axios/dist/node/axios.cjs:2995:11) at IncomingMessage.emit (node:events:529:35) at IncomingMessage.emit (node:domain:552:15) at endReadableNT (node:internal/streams/readable:1368:12) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  {
    "textPayload": "AxiosError: Request failed with status code 503\n    at settle (/workspace/node_modules/axios/dist/node/axios.cjs:1913:12)\n    at IncomingMessage.handleStreamEnd (/workspace/node_modules/axios/dist/node/axios.cjs:2995:11)\n    at IncomingMessage.emit (node:events:529:35)\n    at IncomingMessage.emit (node:domain:552:15)\n    at endReadableNT (node:internal/streams/readable:1368:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {",
    "insertId": "653198de000537eaf5715d09",
    "resource": {
      "type": "cloud_run_revision",
      "labels": {
        "configuration_name": "aggregator",
        "project_id": "**************",
        "location": "us-central1",
        "service_name": "aggregator",
        "revision_name": "aggregator-00009-pox"
      }
    },
    "timestamp": "2023-10-19T21:00:14.341994Z",
    "severity": "ERROR",
    "labels": {
      "instanceId": "0037d6d5d3aabb0489035542a4a56814f64b475a66a0abd00193b52b487c163d72e73e1accb41fa3e805d52ffa0bb9fd3c0a58ae6ea199ec89a4d8b32203c5b285",
      "goog-managed-by": "cloudfunctions"
    },
    "logName": "projects/************/logs/run.googleapis.com%2Fstderr",
    "receiveTimestamp": "2023-10-19T21:00:14.477253017Z",
    "errorGroups": [
      {
        "id": "CNGCx7HHwp2GvQE"
      }
    ]
  }

可能原因及解决方案 #

1. Google RSS地域访问限制 #

你请求的是以色列区域的Google新闻RSS（参数gl=IL），但函数部署在us-central1区域，Google可能会拒绝非以色列IP的请求，返回503错误。

解决方法：

• 将函数部署到以色列区域（il-central1），直接从当地IP发起请求；
• 配置Cloud NAT并关联VPC，让函数通过以色列区域的NAT IP访问目标RSS；
• 临时去掉gl=IL参数，测试通用RSS链接是否能正常访问。

2. 请求头模拟不完整 #

仅添加User-Agent可能不足以绕过Google的反爬机制，缺少Accept、Accept-Language、Referer等真实浏览器请求头会被识别为非合法请求。

解决方法：
模拟完整的浏览器请求头发起请求，示例代码：

export const getKittens = onSchedule("0 * * * *", async (event) => {
    const headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
        'Accept-Language': 'he-IL,he;q=0.8,en-US;q=0.5,en;q=0.3',
        'Referer': 'https://www.google.com/'
    };
    const response = await axios.get("https://news.google.com/rss/search?hl=he&gl=IL&ceid=IL%3Ahe&oc=11&q=kittens", { headers });
    await admin.firestore().collection("news").doc("latest").set({response: response.data});
});

3. 函数网络权限限制 #

如果项目配置了VPC防火墙规则，可能限制了函数的对外HTTPS请求；或者Google RSS域名被出站规则拦截。

解决方法：

• 在Firebase控制台的函数设置中，确认出站网络权限为「允许所有出站流量」；
• 检查VPC防火墙规则，确保允许目标IP或域名的HTTPS请求（端口443）。

4. 函数资源或超时配置不足 #

请求Google RSS可能耗时较长，函数默认内存或超时时间不足会导致请求被中断，表现为503错误。

解决方法：

• 提升函数内存分配（如从256MB调整为512MB），同时设置合理的超时时间：

export const getKittens = onSchedule({
    schedule: "0 * * * *",
    timeoutSeconds: 300, // 设置5分钟超时
    memory: "512MiB"
}, async (event) => {
    const headers = { /* 完整请求头 */ };
    const response = await axios.get("https://news.google.com/rss/search?hl=he&gl=IL&ceid=IL%3Ahe&oc=11&q=kittens", { 
        headers,
        timeout: 10000 // 10秒请求超时
    });
    await admin.firestore().collection("news").doc("latest").set({response: response.data});
});

内容的提问来源于stack exchange，提问作者Nir Gavish