使用Puppeteer的Network.requestIntercepted获取重定向URL遇问题求助

Hey there! Let's break down why your Network.requestIntercepted event isn't triggering and get you those redirect URLs you're after. This is a common gotcha with Puppeteer's CDP (Chrome DevTools Protocol) integration, so let's walk through the fixes step by step.

Common Issues & Solutions #

1. You're Not Properly Initializing the CDP Session & Network Interception #

To use the low-level Network.requestIntercepted event, you need to explicitly enable the Network domain via a CDP session, and configure request interception with the right parameters. A lot of folks skip the critical step of setting the correct interceptionStage or resource patterns.

2. Event Listener is Attached After Setting Interception #

If you set up interception before attaching your event listener, you'll miss the initial requests (including redirects). Always wire up the listener first.

3. Forgetting to Continue Intercepted Requests #

Once you intercept a request, you must tell Chrome to continue (or cancel) it. If you don't, the request will hang, and no further events will fire.

Working Example Code #

Here's a complete, tested snippet that captures redirect URLs using Network.requestIntercepted:

const puppeteer = require('puppeteer');

(async () => {
  // Launch browser (use headless: 'new' for Puppeteer v19+)
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();

  // Create a CDP session for the page
  const cdpSession = await page.target().createCDPSession();
  await cdpSession.send('Network.enable');

  // Attach the event listener FIRST
  cdpSession.on('Network.requestIntercepted', async (event) => {
    console.log(`Original Request URL: ${event.request.url}`);

    // Check for redirect Location header in response
    if (event.responseHeaders) {
      const redirectHeader = event.responseHeaders.find(
        header => header.name.toLowerCase() === 'location'
      );
      if (redirectHeader) {
        console.log(`Redirect Detected: ${redirectHeader.value}`);
      }
    }

    // Critical: Continue the intercepted request
    await cdpSession.send('Network.continueInterceptedRequest', {
      interceptionId: event.interceptionId
    });
  });

  // Configure request interception
  await cdpSession.send('Network.setRequestInterception', {
    patterns: [
      {
        urlPattern: '*', // Match all URLs
        resourceType: 'Document', // Focus on page navigation requests
        interceptionStage: 'HeadersReceived' // Capture after response headers are received
      }
    ]
  });

  // Navigate to a page that redirects (replace with your target URL)
  await page.goto('https://httpbin.org/redirect-to?url=https://example.com');

  await browser.close();
})();

Key Notes to Avoid Pitfalls #

• interceptionStage: Use HeadersReceived if you want to capture redirects—this is when the Location header is available. Using RequestSent won't give you response headers.
• Resource Types: If you want to capture more than just document navigations, adjust the resourceType (e.g., add 'XHR' or use '*' for all resources).
• Puppeteer Version: Ensure you're on a recent version (v19+ recommended) as CDP compatibility can change between releases.
• Don't Mix APIs: If you're using Puppeteer's high-level page.setRequestInterception(true), that uses a different event system (page.on('request', ...)). Stick to either CDP or the high-level API—mixing them can cause conflicts.

Give this setup a try, and you should start seeing those redirect URLs pop up in your console. Let me know if you hit any snags!

内容的提问来源于stack exchange，提问作者turrican_34