Puppeteer无法捕获的Target Closed Error问题求助

问题现象 #

每10到50次请求就会触发以下错误，且错误无法被try/catch捕获，直接导致应用崩溃：

TargetCloseError: Protocol error (Network.getCookies): Target closed
at CallbackRegistry.clear (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:153:36)
at CDPSessionImpl._onClosed (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:468:70)
at Connection.onMessage (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:265:25)
at WebSocket.<anonymous> (/app/node_modules/puppeteer-core/lib/cjs/puppeteer/common/NodeWebSocketTransport.js:62:32)
at callListener (/app/node_modules/ws/lib/event-target.js:290:14)
at WebSocket.onMessage (/app/node_modules/ws/lib/event-target.js:209:9)
at WebSocket.emit (node:events:513:28)
at Receiver.receiverOnMessage (/app/node_modules/ws/lib/websocket.js:1184:20)
at Receiver.emit (node:events:513:28)
at Receiver.dataMessage (/app/node_modules/ws/lib/receiver.js:541:14)
at Receiver.getData (/app/node_modules/ws/lib/receiver.js:459:17)
at Receiver.startLoop (/app/node_modules/ws/lib/receiver.js:158:22)
at Receiver._write (/app/node_modules/ws/lib/receiver.js:84:10)
at writeOrBuffer (node:internal/streams/writable:392:12)
at _write (node:internal/streams/writable:333:10)
at Writable.write (node:internal/streams/writable:337:10) 

现有代码 #

所有涉及Puppeteer的操作均已使用await，代码如下：

let browser = null;
try {
    browser = await puppeteer.launch({
        headless: "new",
        args: [
            `--user-agent=${userAgent}`,
            '--no-sandbox'
        ]
    });
    browser.on('disconnected', () => browserDisconnected = true)
    const page = await browser.newPage();

    await useProxy(page, proxyUrl);
    await page.goto(url);
    await setTimeout(pageLoadTimeout);
    await page.waitForSelector("#Title");

    const renderedContent = await page.content();
    const $ = cheerio.load(renderedContent);

    const title = $("#Title").html();

    output = { title: title };
} catch (error) {
    console.log(error);
    output =  { error: true, error: error.message };
} finally {
    try {
        if(browser !== null) {
            await browser.close();
        }
    } catch(error) {
        console.log("ERROR CLOSING BROWSER:");
        console.log(error);
    }
    return output;
}   

运行环境 #

Google App Engine 中使用 --platform=linux/amd64 node:18 的 Docker 容器

解决方案 #

1. 全局捕获未处理异常 #

该错误属于未处理的Promise拒绝或未捕获异常，需通过Node.js全局事件监听捕获：

// 捕获未处理的Promise拒绝
process.on('unhandledRejection', (reason, promise) => {
  console.error('未处理的Promise拒绝:', reason);
  // 可在此处执行紧急清理，比如关闭存活的浏览器实例
});

// 捕获未捕获的同步异常
process.on('uncaughtException', (error) => {
  console.error('未捕获的异常:', error);
  // 异常后进程可能处于不稳定状态，建议重启
  process.exit(1);
});

2. 监听Puppeteer实例的错误事件 #

错误可能来自Browser或Page的内部事件，需主动监听提前处理：

// 创建浏览器后添加
browser.on('error', (error) => {
  console.error('浏览器实例错误:', error);
  browserDisconnected = true;
});

// 创建页面后添加
page.on('error', (error) => {
  console.error('页面崩溃错误:', error);
  browserDisconnected = true;
});

page.on('pageerror', (error) => {
  console.error('页面内JS错误:', error);
});

3. 修复代码错误并增强校验 #

• 错误的setTimeout使用：await setTimeout(pageLoadTimeout) 不会等待超时，需替换为Puppeteer提供的waitForTimeout：

  await page.waitForTimeout(pageLoadTimeout);
  

• 增加状态校验：在每个Puppeteer操作前检查浏览器连接状态，提前终止流程：

  if (browserDisconnected) throw new Error('浏览器已断开连接');
  await page.goto(url);
  

4. 优化Puppeteer启动参数 #

针对Docker环境添加稳定性参数，解决资源不足或进程通信问题：

browser = await puppeteer.launch({
    headless: "new",
    args: [
        `--user-agent=${userAgent}`,
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-dev-shm-usage', // 解决Docker共享内存不足
        '--disable-gpu',
        '--disable-software-rasterizer',
        '--no-zygote',
        '--single-process' // 减少进程间通信异常
    ],
    timeout: 30000 // 增加启动超时时间
});

5. 检查代理稳定性 #

代理连接不稳定可能导致页面目标断开，可给代理操作添加重试逻辑：

// 给useProxy添加重试封装
async function useProxyWithRetry(page, proxyUrl, retries = 2) {
  try {
    await useProxy(page, proxyUrl);
  } catch (error) {
    if (retries > 0) {
      return useProxyWithRetry(page, proxyUrl, retries - 1);
    }
    throw error;
  }
}

// 替换原调用
await useProxyWithRetry(page, proxyUrl);

6. 考虑使用浏览器池 #

频繁启动关闭浏览器会增加连接异常概率，可使用puppeteer-cluster复用浏览器实例，提升稳定性和性能。

内容的提问来源于stack exchange，提问作者JakZ