You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Puppeteer远程服务器始终加载移动端脚本问题排查

解决无头Chromium爬取时加载移动端脚本的问题

根据你排查出的window.matchMedia判断触发移动端脚本的问题,给出以下几个直接可行的解决思路:

1. 设置桌面级视口尺寸

无头模式下的Chromium默认视口可能偏小,直接触发了移动端媒体查询。在创建页面后添加视口设置:

await page.setViewport({ width: 1920, height: 1080 });

2. 指定桌面端User-Agent

部分网站会结合User-Agent判断设备类型,配合视口设置一起添加桌面UA:

await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36');

3. 强制覆盖目标媒体查询的返回值

如果前两种方法不生效,直接在页面加载前注入脚本,修改matchMedia行为,让移动端查询返回false:

await page.evaluateOnNewDocument(() => {
  const originalMatchMedia = window.matchMedia;
  window.matchMedia = (query: string) => {
    if (query.includes('min-device-width : 320px') && query.includes('max-device-width : 480px')) {
      return {
        matches: false,
        media: query,
        addListener: () => {},
        removeListener: () => {},
        addEventListener: () => {},
        removeEventListener: () => {},
        dispatchEvent: () => false
      };
    }
    return originalMatchMedia(query);
  };
});

4. 检查远程服务器环境配置

  • 确认远程服务器上的Chromium版本与本地一致,旧版本无头模式可能存在默认行为差异;
  • 如果是Docker等容器化部署,检查是否有配置限制了浏览器的设备模拟参数。

修改后的完整代码示例

public async fetchScripts(url: string, waitFor = 'cdn-pipes.js') {
    
    const page = await this.browser.newPage();

    try {
      // 添加桌面视口与UA设置
      await page.setViewport({ width: 1920, height: 1080 });
      await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36');
      // 可选:覆盖matchMedia行为
      await page.evaluateOnNewDocument(() => {
        const originalMatchMedia = window.matchMedia;
        window.matchMedia = (query: string) => {
          if (query.includes('min-device-width : 320px') && query.includes('max-device-width : 480px')) {
            return {
              matches: false,
              media: query,
              addListener: () => {},
              removeListener: () => {},
              addEventListener: () => {},
              removeEventListener: () => {},
              dispatchEvent: () => false
            };
          }
          return originalMatchMedia(query);
        };
      });

      await page.goto(url, {timeout: 10000, waitUntil: 'domcontentloaded'});
      const func = waitFor ? `document.documentElement.innerHTML.indexOf("${waitFor}") !== -1 || document.documentElement.innerHTML.indexOf("spa-detector") !== -1` :
        'document.readyState === "complete"';
      await page.waitForFunction(func, {polling: 500, timeout: 8000}).catch(reason => {
        console.error('page.waitForFunction', {error: reason, url});
      });

      const pageUrls = await page.evaluate(() => {
        const urlArray = Array.from(document.scripts).map((link) => link.src).filter(value => value.includes('taboola.com'));

        return [...new Set(urlArray)];
      });

      console.log('fetchMinimal - urlsArray ', {pageUrls});

      return pageUrls;
    } catch (e) {
      console.error('fetchMinimal - error ', e);
    } finally {
      await page.close();
    }

  }

内容的提问来源于stack exchange,提问作者daniel gi

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.16 03:02:44