本地可运行的Puppeteer爬虫部署Google Cloud Function失败求助
Puppeteer爬虫本地正常,部署到Google Cloud Function加载失败
从Google Cloud日志中仅能获取到如下错误信息:
Function failed on loading user code. This is likely due to a bug in the user code.
部署该函数的步骤如下(同时也尝试过Cloud Function在线编辑器,仍出现相同错误):
- 执行
npm install - 执行
zip -r my-app.zip * - 创建新的Google Cloud Function,配置为:名称
newFunction、内存1GB、运行时Node.js 14、入口函数scrapeFunction - 上传压缩包
index.js
const puppeteer = require('puppeteer'); const { BigQuery } = require('@google-cloud/bigquery'); async function scrapeFunction() { const browser = await puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] }); const page = await browser.newPage(); await page.goto('<URL>', {waitUntil: 'load', timeout: 0}); await page.waitForSelector('span.text'); const info = await page.evaluate(() => { return document.querySelector('span.text').innerText; }); console.log(info); // Write results to BigQuery table const bigqueryClient = new BigQuery(); const dataset = bigqueryClient.dataset('xxx'); const table = dataset.table('yyy'); const rows = [{ info: info }]; await table.insert(rows); await browser.close(); } scrapeFunction();
package.json
{ "name": "newFunction", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC", "dependencies": { "@google-cloud/bigquery": "^6.1.0", "puppeteer": "^19.7.1" } }
内容的提问来源于stack exchange,提问作者dingdongdell
相关产品推荐
相关产品推荐

