NodeJS Lambda + API Gateway超时问题求助(附Puppeteer代码)
问题描述
我有一个基于NodeJS18的Lambda函数,功能是使用Puppeteer打开浏览器生成报告,保存为PDF后上传至S3,最后返回预签名URL。目前遇到以下问题:
- 控制台能打印出有效的预签名URL,但Lambda函数会超时
- 已分配1024MB内存,实际仅使用约700MB,排除资源不足问题
- 出现2-3次超时后,后续调用恢复正常;等待数分钟后再次尝试,又会先出现几次超时,之后恢复正常
- 怀疑是Lambda冷启动或未正确关闭连接导致,但尝试调整后仍未解决
附原函数代码:
const PDFDocument = require("pdf-lib").PDFDocument; const AWS = require("aws-sdk"); const { PutObjectCommand, S3Client } = require("@aws-sdk/client-s3"); const chromium = require("@sparticuz/chromium"); const puppeteer = require("puppeteer-core"); const FRONTEND_URL = process.env.FRONTEND_URL; const BUCKET_NAME = process.env.SLEEP_STUDIES_BUCKET; const lambdaError = (error) => { return { errorType: "InternalServerError", httpStatus: error.statusCode, code: error.statusCode, statusCode: error.statusCode, message: error.message, error, }; }; async function processReport(studyId, scoreSetId, sectionsList, onSpot = false) { const sections = [ "main", ...sectionsList.split(","), ]; console.info("open browser"); const browser = await puppeteer.launch({ executablePath: await chromium.executablePath(), headless: "new", ignoreHTTPSErrors: true, defaultViewport: chromium.defaultViewport, args: [...chromium.args, "--hide-scrollbars", "--disable-web-security"], }); try { const isSecondPageVisible = [ "oxygenSaturationTRT", "arousalTRT", "snoreInTst", "cardiacTST", "movementEvents", "positionStatistics", ].some(key => sections.includes(key)); let trendsPageNumber; if (isSecondPageVisible && sections.includes("note")) { trendsPageNumber = 4; } else if (!isSecondPageVisible && !sections.includes("note")) { trendsPageNumber = 2; } else { trendsPageNumber = 3; } let totalPages = 1; if (isSecondPageVisible) { totalPages++; } if (sections.includes("note")) { totalPages++; } if (sections.includes("trends")) { totalPages++; } let range = `1-${totalPages}`; if (sections.includes("trends")) { range = `1-${totalPages - 1}`; } console.info("generate report"); let report = await generatePdf( browser, studyId, scoreSetId, sections.filter(section => section !== "trends"), range, onSpot, false, totalPages, ); if (sections.includes("trends")) { const landscapePage = await generatePdf( browser, studyId, scoreSetId, "trends", "1", onSpot, true, totalPages, trendsPageNumber, ); console.info("merge reports"); report = await mergePdfs(report, landscapePage); } console.info("save report"); const fileUrl = await savePdf(studyId, scoreSetId, report, onSpot); console.info("close browser"); browser.close(); console.info("browser closed, returning response for filename", fileUrl); return { message: "Report generated and uploaded to S3!", fileUrl, }; } catch (e) { console.info(e); browser.close(); console.info("browser closed, returning error"); e.statusCode = e.statusCode ?? 500; throw e; } } async function generatePdf( browser, studyId, scoreSetId, sections, range, watermark, landscape, totalPages = 4, trendsPageNumber = 4, ) { try { console.info("open new page for ", sections.toString()); const page = await browser.newPage(); let url = `${FRONTEND_URL}/report/${studyId}/${scoreSetId}/print?sections=${sections}&totalPages=${totalPages}&trendsPageNumber=${trendsPageNumber}`; if (watermark) { url += `&watermark=${watermark}`; } console.info("navigate to page", url); await page.goto(url, { waitUntil: "networkidle0" }); await page.addStyleTag({ content: ` @page { size: A4 ${landscape ? "landscape" : "portrait"} !important; margin: 0 !important; } .navbar { display: none; } `, }); await page.emulateMediaType("screen"); console.info("generate report for pages ", range); return await page.pdf({ margin: "none", footerTemplate: "", printBackground: true, format: "A4", landscape, pageRanges: range, }); } catch (e) { console.error(e); throw e; } } async function mergePdfs(portraitPages, landscapePage) { try { const pdfsToMerge = [portraitPages, landscapePage]; const mergedPdf = await PDFDocument.create(); for (const pdfBytes of pdfsToMerge) { const pdf = await PDFDocument.load(pdfBytes); const copiedPages = await mergedPdf.copyPages(pdf, pdf.getPageIndices()); copiedPages.forEach((page) => { mergedPdf.addPage(page); }); } return await mergedPdf.save(); } catch (e) { console.info(e); throw e; } } async function savePdf(studyId, scoreSetId, buffer, onSpot = false) { const path = `${studyId}/reports/${scoreSetId}/${onSpot ? "draft-" : ""}report-${new Date().getTime() / 1000}.pdf`; const client = new S3Client({}); const command = new PutObjectCommand({ Bucket: BUCKET_NAME, Key: path, Body: buffer, }); await client.send(command); console.log("file uploaded on s3", path); if (!onSpot) { return path; } const s3 = new AWS.S3(); const params = { Bucket: BUCKET_NAME, Key: path, }; return await s3.getSignedUrlPromise("getObject", params); } exports.handler = function (event, context, callback) { if (!FRONTEND_URL) { const error = new Error(); error.message = "Frontend URL not found!"; error.statusCode = 500; throw error; } if (!BUCKET_NAME) { const error = new Error(); error.message = "Required S3 Environment Variables aren't found!"; error.statusCode = 500; throw error; } // @Todo: extract authorization token and pass it as query param in FE const { sections, onSpot, sleep_study_id, score_set_id } = event; context.callbackWaitsForEmptyEventLoop = false; processReport(sleep_study_id, score_set_id, sections ?? "", onSpot) .then((response) => { callback(null, response); }) .catch((e) => { console.log(e); e.statusCode = e.statusCode ?? 500; callback(lambdaError(e)); }); };
核心问题分析
这种"冷启动后前几次超时,后续正常"的现象,本质是Lambda执行环境初始化(尤其是Chromium/Puppeteer启动)的开销在冷启动阶段超出了超时阈值,同时存在异步资源未正确释放导致的事件循环残留问题。
解决方案
1. 确保浏览器实例异步关闭
当前代码中browser.close()是异步操作,但未使用await等待完成,可能导致Lambda在浏览器未完全关闭时就结束,残留的异步任务会触发超时。
修改processReport函数中的关闭逻辑:
// try块中 console.info("close browser"); await browser.close(); // 添加await console.info("browser closed, returning response for filename", fileUrl); // catch块中 console.info(e); await browser.close(); // 添加await console.info("browser closed, returning error");
2. 优化Puppeteer启动与页面加载
冷启动时Chromium的初始化是主要耗时点,同时页面加载的等待策略可能过于严格:
- 调整
page.goto的waitUntil参数:networkidle0需要等待所有网络连接断开,若页面存在长连接会导致等待超时,改为networkidle2(仅允许2个网络连接)或domcontentloaded - 简化Chromium启动参数,禁用不必要的功能减少启动时间
修改generatePdf中的页面加载逻辑:
await page.goto(url, { waitUntil: "networkidle2" }); // 替换networkidle0
调整Puppeteer启动参数:
const browser = await puppeteer.launch({ executablePath: await chromium.executablePath(), headless: "new", ignoreHTTPSErrors: true, defaultViewport: chromium.defaultViewport, args: [ ...chromium.args, "--hide-scrollbars", "--disable-web-security", "--disable-extensions", // 禁用扩展 "--disable-dev-shm-usage", // 避免/dev/shm空间不足 "--no-sandbox" // Lambda环境下可加速启动 ], });
3. 统一AWS SDK版本,避免连接泄漏
当前代码同时使用了@aws-sdk/client-s3(v3)和旧版AWS.S3(v2),两者的连接池管理可能冲突,导致连接未正确释放。改用v3的s3-request-presigner生成预签名URL:
首先安装依赖(Lambda层或打包时包含):
npm install @aws-sdk/s3-request-presigner
修改savePdf函数:
const { PutObjectCommand, S3Client, GetObjectCommand } = require("@aws-sdk/client-s3"); const { getSignedUrl } = require("@aws-sdk/s3-request-presigner"); // 新增 async function savePdf(studyId, scoreSetId, buffer, onSpot = false) { const path = `${studyId}/reports/${scoreSetId}/${onSpot ? "draft-" : ""}report-${new Date().getTime() / 1000}.pdf`; const client = new S3Client({}); const command = new PutObjectCommand({ Bucket: BUCKET_NAME, Key: path, Body: buffer, }); await client.send(command); console.log("file uploaded on s3", path); if (!onSpot) { return path; } // 使用v3 SDK生成预签名URL const getCommand = new GetObjectCommand({ Bucket: BUCKET_NAME, Key: path, }); return await getSignedUrl(client, getCommand, { expiresIn: 3600 }); // 设置过期时间 }
4. 调整Lambda超时配置
冷启动阶段Chromium初始化+页面加载的总耗时可能超过Lambda默认的30秒超时,建议将Lambda函数的超时时间调整为60秒(根据实际测试调整),给冷启动足够的初始化时间。
5. 复用Lambda执行环境(可选)
Lambda的执行环境会在调用后保留一段时间,可通过复用浏览器实例减少冷启动开销。但需注意处理实例状态,避免多请求冲突:
// 全局变量存储浏览器实例 let browserInstance = null; async function getBrowser() { if (!browserInstance) { browserInstance = await puppeteer.launch({ /* 启动参数 */ }); // 监听浏览器关闭事件,清空实例 browserInstance.on("disconnected", () => { browserInstance = null; }); } return browserInstance; } // 在processReport中使用 const browser = await getBrowser();
内容的提问来源于stack exchange,提问作者Vaduva Mihaita Bogdan

