如何让Puppeteer与Firebase Functions兼容（Node.js 16及新版Puppeteer）

在Node.js16版本的Firebase Functions中使用Puppeteer抓取网页时，函数调用崩溃，Google Cloud日志提示找不到Chromium（版本1069273），此前基于Node.js8的解决方案已失效，需适配Node.js16的解决办法。

原代码 #

index.ts #

import * as functions from "firebase-functions";
import { scrapeWebsite } from "./core/scraping";

exports.scrape = functions
  .runWith({ timeoutSeconds: 120, memory: "2GB" })
  .region("us-central1")
  .https.onRequest(async (request, response) => {
    const stories = await scrapeWebsite();
    response.type("json").send(stories);
  });

scraping模块（./core/scraping.ts） #

import puppeteer from "puppeteer";

const scrapeWebsite = async () => {
  let stories: any = [];
  const browser = await puppeteer.launch({
    headless: true,
    timeout: 20000,
    ignoreHTTPSErrors: true,
    slowMo: 0,
    args: [
      "--disable-gpu",
      "--disable-dev-shm-usage",
      "--disable-setuid-sandbox",
      "--no-first-run",
      "--no-sandbox",
      "--no-zygote",
      "--window-size=1280,720",
    ],
  });

  try {
    const page = await browser.newPage();

    await page.setViewport({ width: 1280, height: 720 });

    // Block images, videos, fonts from downloading
    await page.setRequestInterception(true);

    page.on("request", (interceptedRequest) => {
      const blockResources = ["script", "stylesheet", "image", "media", "font"];
      if (blockResources.includes(interceptedRequest.resourceType())) {
        interceptedRequest.abort();
      } else {
        interceptedRequest.continue();
      }
    });

    // Change the user agent of the scraper
    await page.setUserAgent(
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"
    );

    await page.goto("https://www.nytimes.com/", {
      waitUntil: "networkidle0",
    });

    const storySelector = "section.story-wrapper h3";

    // Only get the top 10 headlines
    stories = await page.$$eval(storySelector, (divs: any) =>
      divs
        .slice(0, 10)
        .map(
          (div: HTMLElement, index: number) => `${index + 1}. ${div.innerText}`
        )
    );
  } catch (error) {
    return error;
  } finally {
    if (browser) {
      await browser.close();
    }
  }
  return stories;
};

export { scrapeWebsite };

错误日志 #

Error: Could not find Chromium (rev. 1069273). This can occur if either
you did not perform an installation before running the script (e.g. npm install) or
your cache path is incorrectly configured (which is: /home/sysadm/.cache/puppeteer).
For (2), check out our guide on configuring puppeteer.
at ChromeLauncher.resolveExecutablePath (/var/www/html/oas-restapi-fork1/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ProductLauncher.js:119:27)
at ChromeLauncher.executablePath (/var/www/html/oas-restapi-fork1/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ChromeLauncher.js:205:25)
at ChromeLauncher.launch (/var/www/html/oas-restapi-fork1/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ChromeLauncher.js:93:37)

步骤1：替换依赖 #

卸载原puppeteer，安装puppeteer-core和chrome-aws-lambda——后者专门为Serverless环境（包括Firebase Functions）提供兼容的Chromium二进制文件：

npm uninstall puppeteer
npm install puppeteer-core chrome-aws-lambda --save

步骤2：修改scraping模块代码 #

调整代码以使用chrome-aws-lambda提供的Chromium路径，并适配Serverless环境：

import puppeteer from "puppeteer-core";
import chrome from "chrome-aws-lambda";

const scrapeWebsite = async () => {
  let stories: any = [];
  let browser: puppeteer.Browser | null = null;

  try {
    // 获取适配Serverless环境的Chromium路径和启动参数
    const executablePath = await chrome.executablePath;
    browser = await puppeteer.launch({
      executablePath: executablePath,
      headless: chrome.headless,
      timeout: 20000,
      ignoreHTTPSErrors: true,
      args: [
        ...chrome.args,
        "--disable-dev-shm-usage",
        "--no-sandbox",
        "--window-size=1280,720",
      ],
    });

    const page = await browser.newPage();

    await page.setViewport({ width: 1280, height: 720 });

    // Block images, videos, fonts from downloading
    await page.setRequestInterception(true);

    page.on("request", (interceptedRequest) => {
      const blockResources = ["script", "stylesheet", "image", "media", "font"];
      if (blockResources.includes(interceptedRequest.resourceType())) {
        interceptedRequest.abort();
      } else {
        interceptedRequest.continue();
      }
    });

    // Change the user agent of the scraper
    await page.setUserAgent(
      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"
    );

    await page.goto("https://www.nytimes.com/", {
      waitUntil: "networkidle0",
    });

    const storySelector = "section.story-wrapper h3";

    // Only get the top 10 headlines
    stories = await page.$$eval(storySelector, (divs: any) =>
      divs
        .slice(0, 10)
        .map(
          (div: HTMLElement, index: number) => `${index + 1}. ${div.innerText}`
        )
    );
  } catch (error) {
    return error;
  } finally {
    if (browser) {
      await browser.close();
    }
  }
  return stories;
};

export { scrapeWebsite };

步骤3：确认Firebase Functions配置 #

确保函数的运行配置满足Chromium需求：

• 内存至少设置为1GB（代码中已设为2GB，符合要求）
• 超时时间设置为足够长（代码中已设为120秒，符合要求）

错误原因说明 #

原puppeteer包默认会在本地缓存目录安装Chromium，但Firebase Functions部署时不会包含本地缓存的二进制文件，导致函数运行时找不到Chromium。而chrome-aws-lambda会将适配Serverless环境的Chromium二进制文件打包进部署包，同时提供对应的启动参数，完美适配Node.js16版本的Firebase Functions环境。

内容的提问来源于stack exchange，提问作者Dumitru