如何在Puppeteer中保存Cookie并跨会话加载以避免重复请求页面？

Got it, let's fix that redundant page load by persisting cookies between Puppeteer sessions! The core idea is to capture the cookies set during your first visit to the site, save them, then inject them into a new session before navigating. This way, the second session already has all the necessary cookies the site expects, so you won't need to reload the page twice.

Step-by-Step Breakdown #

• Capture Cookies in First Session: After your initial page load, extract the cookies from the page.
• Save Cookies: Store them (we'll use a JSON file for persistence, but you could also pass them in-memory if running sessions in the same script).
• Load Cookies in Second Session: Before navigating to the page in your new session, inject the saved cookies.

Full Code Example #

First, run this script to capture and save the cookies:

const puppeteer = require('puppeteer');
const fs = require('fs');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  
  // Visit the page once to get the necessary cookies
  await page.goto('https://www.genecards.org/cgi-bin/carddisp.pl?gene=BSCL2');
  
  // Extract all cookies for the site
  const cookies = await page.cookies();
  
  // Save cookies to a JSON file
  fs.writeFileSync('genecards_cookies.json', JSON.stringify(cookies, null, 2));
  
  await browser.close();
})();

Then, use this script to reuse the saved cookies in a new session:

const puppeteer = require('puppeteer');
const fs = require('fs');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  
  // Load cookies from the saved file
  const savedCookies = JSON.parse(fs.readFileSync('genecards_cookies.json', 'utf8'));
  
  // Inject the cookies into the new page
  await page.setCookie(...savedCookies);
  
  // Navigate to the page - no need for a second load, cookies are already present!
  await page.goto('https://www.genecards.org/cgi-bin/carddisp.pl?gene=BSCL2');
  
  // Perform your desired action
  const linkHandlers = await page.$x("//div[@id='enhancers']//a[@data-track-event='Table See-All']");
  if (linkHandlers.length > 0) {
    const [response] = await Promise.all([
      page.waitForResponse(response => response.url().includes('/gene/api/data/Enhancers')),
      linkHandlers[0].click()
    ]);
    const resp_text = await response.text();
    console.log(resp_text);
  } else {
    throw new Error("Link not found");
  }
  
  await browser.close();
})();

Key Notes #

• Domain Matching: Puppeteer's page.cookies() captures all cookies tied to the current page's domain, so you don't have to worry about incorrect domains when injecting.
• In-Memory Alternative: If you want to run both sessions in the same script (no file needed), just pass the cookies variable directly from the first session to the second instead of writing to a file.
• Cookie Expiry: Keep in mind that some cookies may have short expiry times, so you'll need to refresh them periodically if you're reusing them over a longer period.

内容的提问来源于stack exchange，提问作者user1424739