AWS Elastic Beanstalk中使用PlaywrightCrawler报错求助

核心原因分析 #

• AWS Elastic Beanstalk部署流程会清理用户缓存目录（如/root/.cache或/home/chrome/.cache），导致Playwright预安装的浏览器文件丢失
• Playwright版本与镜像/环境中预装的浏览器版本不匹配，引发路径不对应问题
• Alpine镜像的musl libc与Playwright Chromium的glibc依赖存在兼容性冲突
• 部署命令执行时机错误，浏览器未正确安装到运行环境

针对Beanstalk原生环境（非Docker）的解决方案 #

1. 指定浏览器安装到应用目录
   设置环境变量PLAYWRIGHT_BROWSERS_PATH=./playwright-browsers（可通过Beanstalk控制台或.ebextensions/env.config配置），让浏览器安装到应用目录下（不会被清理）。在.ebextensions/playwright.config中添加容器命令：

   container_commands:
     01_install_playwright:
       command: "npx playwright install --with-deps chromium"
       env:
         PLAYWRIGHT_BROWSERS_PATH: "./playwright-browsers"
   

2. 修复文件权限
   添加权限修复命令，确保应用运行用户能访问浏览器文件：

   container_commands:
     02_fix_permissions:
       command: "chmod -R 755 ./playwright-browsers"
   

3. 代码中手动指定可执行路径
   在PlaywrightCrawler配置中明确指定浏览器路径：

   const { PlaywrightCrawler } = require('apify');
   const { chromium } = require('playwright');
   
   const crawler = new PlaywrightCrawler({
     launchContext: {
       launcher: chromium,
       launchOptions: {
         executablePath: './playwright-browsers/chromium-1033/chrome-linux/chrome',
         headless: true,
         args: ['--no-sandbox', '--disable-dev-shm-usage']
       }
     },
     // 其他爬虫配置
   });
   

针对Docker镜像（zenika/alpine-chrome）的解决方案 #

1. 对齐Playwright版本
   镜像预装的Playwright版本可能与项目依赖不一致，导致下载的浏览器版本不匹配。在Dockerfile中明确安装与项目package.json一致的Playwright版本：

   FROM zenika/alpine-chrome:with-playwright
   WORKDIR /app
   COPY package*.json ./
   RUN npm install
   # 替换<你的项目Playwright版本>为实际版本号，比如1.38.0
   RUN npx playwright@<你的项目Playwright版本> install chromium --with-deps
   COPY . .
   CMD ["npm", "start"]
   

2. 改用兼容性更好的官方镜像
   Alpine的musl libc易引发兼容问题，建议改用官方Playwright Debian镜像：

   FROM mcr.microsoft.com/playwright:v1.38.0-focal
   WORKDIR /app
   COPY package*.json ./
   RUN npm install
   COPY . .
   CMD ["npm", "start"]
   

   该镜像已预装所有依赖和浏览器，无需额外执行npx playwright install。

手动排查步骤 #

• 登录Beanstalk实例或Docker容器，检查浏览器路径是否存在：

  # 原生环境
  ls /root/.cache/ms-playwright/chromium-1033/chrome-linux/
  # Docker环境
  ls /home/chrome/.cache/ms-playwright/chromium-1028/chrome-linux/
  

• 执行带日志的安装命令，排查下载或权限错误：

  npx playwright install chromium --verbose
  

内容的提问来源于stack exchange，提问作者Shahryar Faraji