如何在Docker容器中运行Puppeteer与Node.js爬虫?
问题:Puppeteer+Node.js爬虫容器化失败,浏览器启动报错
我用Puppeteer和Node.js写了一个爬虫,现在想把它容器化,但启动浏览器爬取时一直出问题。
我当前的基础Dockerfile(没包含Puppeteer相关依赖,试过加Chrome、Puppeteer等各种修改都没用):
# Use Node.js runtime as the base image FROM node:18 # Set the working directory in the container WORKDIR /usr/src/app # Copy package.json and package-lock.json to the working directory COPY package*.json ./ # Install dependencies RUN npm install # Copy the rest of the application code COPY . . # Expose the port the app runs on EXPOSE 8080 # Command to run the application CMD ["node", "scraper.js"]
启动浏览器的代码片段:
// Launch browser const browser = await launch({ headless: true, defaultViewport: null });
我试过几种解决方法,但都无效,现在需要帮忙解决这个问题,让爬虫正常运行。
遇到的错误:
爬取过程中发生错误:
Error: Failed to launch the browser process! web-crawler-1 | rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2 TROUBLESHOOTING: https://pptr.dev/troubleshooting at Interface.onClose (file:///usr/src/app/node_modules/@puppeteer/browsers/lib/esm/launch.js:301:24) at Interface.emit (node:events:529:35) at Interface.close (node:internal/readline/interface:534:10) at Socket.onend (node:internal/readline/interface:260:10) at Socket.emit (node:events:529:35) at endReadableNT (node:internal/streams/readable:1400:12) at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
解决方案
这个错误是架构不兼容导致的:你用的node:18镜像如果是ARM架构(比如M系列Mac),但Puppeteer下载的Chrome是x86_64架构,Rosetta转译时找不到对应链接器;反之在x86机器上跑ARM镜像也会触发该问题。下面是两种可行的修复方案:
方案1:使用Puppeteer官方镜像(最省心)
直接用Puppeteer官方维护的镜像,已经预装了兼容的Chrome和所有依赖,彻底规避架构、权限问题:
# 使用Puppeteer官方镜像,内置Node.js和兼容的Chrome FROM puppeteer/puppeteer:21.6.0 WORKDIR /usr/src/app # 复制依赖文件 COPY package*.json ./ # 以puppeteer用户身份安装依赖,避免权限问题 RUN npm install # 复制爬虫代码 COPY . . # 切换到非root用户运行,解决Chrome沙箱权限限制 USER puppeteer # 启动命令 CMD ["node", "scraper.js"]
同时修改浏览器启动代码,加上容器环境必须的参数:
const browser = await launch({ headless: 'new', // Puppeteer 19+推荐使用新版无头模式 defaultViewport: null, args: [ '--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage', // 解决容器/dev/shm内存不足问题 '--disable-gpu' // 无头模式无需GPU加速 ] });
方案2:手动在Node镜像中安装兼容Chrome
如果不想用官方镜像,可以针对对应架构手动安装Chrome:
x86_64架构Dockerfile
FROM node:18-bullseye WORKDIR /usr/src/app # 安装Chrome及依赖 RUN apt-get update && apt-get install -y \ chromium \ chromium-driver \ && rm -rf /var/lib/apt/lists/* # 告诉Puppeteer使用系统安装的Chrome ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium COPY package*.json ./ RUN npm install COPY . . CMD ["node", "scraper.js"]
ARM架构(如M1/M2 Mac)Dockerfile
FROM node:18-bullseye WORKDIR /usr/src/app # ARM架构安装chromium RUN apt-get update && apt-get install -y \ chromium \ chromium-driver \ && rm -rf /var/lib/apt/lists/* ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium COPY package*.json ./ RUN npm install COPY . . CMD ["node", "scraper.js"]
同样需要修改启动代码,加上方案1中的args参数。
额外注意事项
- 如果是M系列Mac,确保Docker开启Rosetta转译(Docker设置->Features in development中勾选"Use Rosetta for x86/amd64 emulation on Apple Silicon"),或直接构建ARM架构镜像。
- 不要使用
node:18-alpine镜像,Alpine的musl libc与Chrome兼容性极差,会引发更多问题。
内容的提问来源于stack exchange,提问作者Parv
相关产品推荐
相关产品推荐

