Next.js Puppeteer爬虫部署至Vercel后出现500内部服务器错误
Next.js + Playwright 爬虫部署至Vercel后出现500内部服务器错误问题
我正在开发基于Next.js和Puppeteer的网页爬虫,本地运行完全正常,但部署到Vercel后,调用Puppeteer时出现500内部服务器错误。我查阅了无服务器Puppeteer函数部署至Vercel的相关指南,尝试改用Playwright,但部署后依旧无法正常工作。
相关代码片段如下:
import chromium from "chrome-aws-lambda"; import playwright from "playwright-core"; //FUNCTION TO RUN SEPARATE SCRAPE FUNCTIONS async function scrapeInfiniteScrollItems(page, getNews, src) { let items = {}; try { items = await page.evaluate(getNews); } catch (e) { console.log(e); console.log("bad source", src); } return items; } //FUNCTION TO SET UP BROWSER AND RETURN export default async function handler(req, res) { const browser = await playwright.chromium.launch({ args: chromium.args, executablePath: process.env.NODE_ENV !== "development" ? await chromium.executablePath : "/usr/bin/chromium", headless: process.env.NODE_ENV !== "development" ? chromium.headless : true, }); const page = await browser.newPage(); page.setJavaScriptEnabled(false); page.setViewport({ width: 1280, height: 3000 }); await page.goto("https://www.foxnews.com/politics"); let items = await scrapeInfiniteScrollItems(page, extractFox, "fox"); //NOTE: I didn't include the extractFox function because it didnt use any puppeteer functions allArticles.fox = items; await browser.close(); res.status(200).json(allArticles); }
我还尝试过其他方案,比如使用基于废弃Node版本的示例项目,以及当前正在尝试的截图方案,但均未解决问题。我猜测需要安装适配Vercel无服务器环境的替代库来解决该问题。
内容的提问来源于stack exchange,提问作者denniswang
相关产品推荐
相关产品推荐

