如何使用Puppeteer下载带认证查询字符串的图片？

核心问题原因 #

page.goto() 是用于导航到HTML文档类资源的方法，直接访问图片这类二进制资源会触发"无效URL"错误——因为浏览器无法将图片解析为可导航的页面内容。

可行解决方案 #

方案1：利用Puppeteer的下载监听功能 #

配置页面自动处理下载请求，通过模拟点击触发下载：

const puppeteer = require('puppeteer');
const fs = require('fs');
const path = require('path');

(async () => {
  const browser = await puppeteer.launch({
    executablePath: '/usr/bin/brave-browser',
    headless: false,
    defaultViewport: null
  });
  const page = await browser.newPage();

  // 创建下载目录并配置自动下载行为
  const downloadDir = path.resolve(__dirname, 'downloads');
  if (!fs.existsSync(downloadDir)) fs.mkdirSync(downloadDir);
  await page._client.send('Page.setDownloadBehavior', {
    behavior: 'allow',
    downloadPath: downloadDir
  });

  // 模拟点击下载图片
  await page.evaluate((imgUrl) => {
    const link = document.createElement('a');
    link.href = imgUrl;
    link.download = '';
    document.body.appendChild(link);
    link.click();
    document.body.removeChild(link);
  }, 'https://s3.amazonaws.com/spypoint-production-account-failover/5c9f7dc7267fc300f968bb01/60632085b8683500145706d1/20230213/PICT1853_202302131500FHth3.jpg?X-Amz-Expires=86400&X-Amz-Date=20230401T094532Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATVANQEDJ5KPEZXK2%2F20230401%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=e9be25f01b577c1116b1c5ee00e0a1c1386b716cc7fe0ea19cd4aad77d61684d');

  // 等待下载完成（可根据文件大小调整等待时长）
  await new Promise(resolve => setTimeout(resolve, 3000));

  await browser.close();
})();

方案2：直接用Node.js HTTP模块请求签名URL #

既然签名URL已包含完整认证信息，无需依赖浏览器上下文，直接用Node.js原生模块下载更高效：

const https = require('https');
const fs = require('fs');
const path = require('path');

const imgUrl = 'https://s3.amazonaws.com/spypoint-production-account-failover/5c9f7dc7267fc300f968bb01/60632085b8683500145706d1/20230213/PICT1853_202302131500FHth3.jpg?X-Amz-Expires=86400&X-Amz-Date=20230401T094532Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATVANQEDJ5KPEZXK2%2F20230401%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=e9be25f01b577c1116b1c5ee00e0a1c1386b716cc7fe0ea19cd4aad77d61684d';
const savePath = path.resolve(__dirname, 'downloads', 'PICT1853.jpg');

const fileStream = fs.createWriteStream(savePath);
https.get(imgUrl, (res) => {
  res.pipe(fileStream);
  fileStream.on('finish', () => {
    fileStream.close();
    console.log('图片下载完成');
  });
}).on('error', (err) => {
  fs.unlinkSync(savePath); // 删除未完成的文件
  console.error('下载失败:', err.message);
});

方案3：拦截Puppeteer资源请求获取图片内容 #

如果必须依赖浏览器会话（比如需要特定Cookie），可拦截图片请求并保存二进制内容：

const puppeteer = require('puppeteer');
const fs = require('fs');

(async () => {
  const browser = await puppeteer.launch({
    executablePath: '/usr/bin/brave-browser',
    headless: false
  });
  const page = await browser.newPage();

  // 开启请求拦截
  await page.setRequestInterception(true);
  page.on('request', (req) => {
    if (req.url().includes('PICT1853_202302131500FHth3.jpg')) {
      req.continue().then(async () => {
        const res = await req.response();
        const imgBuffer = await res.buffer();
        fs.writeFileSync('PICT1853.jpg', imgBuffer);
        console.log('图片保存完成');
      });
    } else {
      req.continue();
    }
  });

  // 导航到包含目标图片的私有空间页面
  await page.goto('https://your-private-space-page-url'); // 替换为实际私有空间页面URL

  await browser.close();
})();

关键注意事项 #

• 签名URL有过期时间（示例中为24小时），确保使用时URL仍处于有效期内
• 无头模式下必须配置Page.setDownloadBehavior，否则下载会被阻塞
• 直接用Node.js请求的方式性能最优，无需启动浏览器，适合批量下载场景

内容的提问来源于stack exchange，提问作者Savend