爬虫爬取网站时遭遇旧版IE浏览器不兼容提示的问题求助

架构老林

2026-8-22

问题描述

尝试爬取网站https://iltacon2022.expofp.com/时，持续收到以下检测提示：

if (window.navigator.userAgent.indexOf("Trident/") !== -1) {
    alert("Your are using old unsupported Internet Explorer browser.
Please upgrade to view this page properly.")

已尝试使用selenium和requests模块，但两种方式均出现相同问题。

代码示例

from bs4 import BeautifulSoup as bs
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
import random
import requests

options = Options()
options.headless = False
driver = webdriver.Firefox(options=options)

url = "https://iltacon2022.expofp.com/"

driver.get(url)

time.sleep(6)

soup = bs(driver.page_source, 'lxml')

driver.quit()

print(soup)

输出内容

<html lang="en"><head>
<meta charset="utf-8"/>
<link href="https://iltacon2022.expofp.com/packages/master/favicon.png" rel="shortcut icon"/>
<meta content="user-scalable=no, initial-scale=1.0, maximum-scale=1.0, width=device-width" name="viewport"/>
<!-- <meta name="theme-color" content="#000000" /> -->
<title>ILTACON2022 – Gaylord National Resort and Convention Center | August 22–25, 2022 | Monday – Thursday – Expo Floor Plan by ExpoFP</title>
<script>
            if (window.navigator.userAgent.indexOf("Trident/") !== -1) {
                alert("Your are using old unsupported Internet Explorer browser.
Please upgrade to view this page properly.");
            }
        </script>
<style>
            html,
            body {
                touch-action: none;
                margin: 0;
                padding: 0;
                height: 100%;
                width: 100%;
                background: #ebebeb;
                position: fixed;
                overflow: hidden;
            }
            @media (max-width: 820px) and (min-width: 500px) {
                html {
                    font-size: 13px;
                }
            }
        </style>
<style>
            .lds-grid {
                top: 42vh;
                margin: 0 auto;
                display: block;
                position: relative;
                width: 64px;
                height: 64px;
            }

            .lds-grid div {
                position: absolute;
                width: 13px;
                height: 13px;
                background: #aaa;
                border-radius: 50%;
                /* border: solid 1px #fff; */
                animation: lds-grid 1.2s linear infinite;
            }

            .lds-grid div:nth-child(1) {
                top: 6px;
                left: 6px;
                animation-delay: 0s;
            }

            .lds-grid div:nth-child(2) {
                top: 6px;
                left: 26px;
                animation-delay: -0.4s;
            }

            .lds-grid div:nth-child(3) {
                top: 6px;
                left: 45px;
                animation-delay: -0.8s;
            }

            .lds-grid div:nth-child(4) {
                top: 26px;
                left: 6px;
                animation-delay: -0.4s;
            }

            .lds-grid div:nth-child(5) {
                top: 26px;
                left: 26px;
                animation-delay: -0.8s;
            }

            .lds-grid div:nth-child(6) {
                top: 26px;
                left: 45px;
                animation-delay: -1.2s;
            }

            .lds-grid div:nth-child(7) {
                top: 45px;
                left: 6px;
                animation-delay: -0.8s;
            }

            .lds-grid div:nth-child(8) {
                top: 45px;
                left: 26px;
                animation-delay: -1.2s;
            }

            .lds-grid div:nth-child(9) {
                top: 45px;
                left: 45px;
                animation-delay: -1.6s;
            }

            @keyframes lds-grid {
                0%,
                100% {
                    opacity: 1;
                }

                50% {
                    opacity: 0.5;
                }
            }
        </style>
<link as="script" href="https://iltacon2022.expofp.com/data/data.js" rel="preload"/>
<link as="script" href="https://iltacon2022.expofp.com/data/fp.svg.js" rel="preload"/>
<link as="script" href="https://iltacon2022.expofp.com/packages/master/floorplan.js" rel="preload"/>
<link as="script" href="https://iltacon2022.expofp.com/packages/master/vendors~floorplan.js" rel="preload"/>
<link as="style" href="https://iltacon2022.expofp.com/packages/master/vendor/fa/css/fontawesome-all.min.css" rel="preload"/>
<link as="style" href="https://iltacon2022.expofp.com/packages/master/vendor/sanitize-css/sanitize.css" rel="preload"/>
<link as="style" href="https://iltacon2022.expofp.com/packages/master/vendor/perfect-scrollbar/css/perfect-scrollbar.css" rel="preload"/>
<!-- Fonts are anonymous because those will be loaded with FontFace -->
<link as="font" crossorigin="anonymous" href="https://iltacon2022.expofp.com/packages/master/vendor/fa/webfonts/fa-regular-400.woff2" rel="preload"/>
<link as="font" crossorigin="anonymous" href="https://iltacon2022.expofp.com/packages/master/vendor/fa/webfonts/fa-solid-900.woff2" rel="preload"/>
<link as="font" crossorigin="anonymous" href="https://iltacon2022.expofp.com/packages/master/vendor/fa/webfonts/fa-light-300.woff2" rel="preload"/>
<link as="font" crossorigin="anonymous" href="https://iltacon2022.expofp.com/packages/master/fonts/oswald-v17-cyrillic_latin-500.woff2" rel="preload"/>
<link as="font" crossorigin="anonymous" href="https://iltacon2022.expofp.com/packages/master/fonts/oswald-v17-cyrillic_latin-300.woff2" rel="preload"/>
<script src="https://iltacon2022.expofp.com/data/data.js"></script><script src="https://iltacon2022.expofp.com/data/wf.data.js"></script><script src="https://iltacon2022.expofp.com/data/fp.svg.js"></script><script charset="utf-8" src="https://iltacon2022.expofp.com/packages/master/vendors~floorplan.js"></script><script charset="utf-8" src="https://iltacon2022.expofp.com/packages/master/floorplan.js"></script></head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div class="expofp-floorplan" data-event-id="iltacon2022"><div></div></div>
<script src="https://iltacon2022.expofp.com/packages/master/expofp.js"></script>
</body></html>

解决方案

关键问题分析

页面中的IE检测脚本仅针对Trident内核（IE浏览器）弹出提示，你的请求使用的是Firefox或默认requests UA，不会触发该提示，因此它不是爬取失败的原因。
该网站是单页应用（SPA），核心的展位、参展商数据是通过JavaScript动态渲染的，直接获取静态HTML或短时间等待后抓取的源码，无法得到实际内容。

方案一：直接抓取API数据

页面的核心数据直接暴露在公开的JS文件中，无需渲染页面即可获取：

import requests
import re
import json

def extract_js_data(url):
    response = requests.get(url)
    # 匹配JS文件中类似window.data = {...}的结构
    pattern = re.compile(r'window\.[a-z]+ = (.*?);', re.DOTALL)
    match = pattern.search(response.text)
    return json.loads(match.group(1)) if match else None

# 获取参展商核心数据
core_data = extract_js_data("https://iltacon2022.expofp.com/data/data.js")
# 获取额外展位数据
extra_data = extract_js_data("https://iltacon2022.expofp.com/data/wf.data.js")

# 示例：打印所有参展商名称
if core_data:
    print("参展商列表：")
    for exhibitor in core_data['exhibitors']:
        print(exhibitor['name'])

方案二：优化Selenium等待逻辑

如果必须通过页面渲染获取内容，需等待JavaScript完全加载完成，而非固定等待时长：

from bs4 import BeautifulSoup as bs
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.headless = False
driver = webdriver.Firefox(options=options)

url = "https://iltacon2022.expofp.com/"
driver.get(url)

# 等待核心元素加载（这里以参展商名称元素为例，可根据实际调整）
try:
    WebDriverWait(driver, 15).until(
        EC.presence_of_element_located(("css selector", ".expofp-exhibitor-name"))
    )
except Exception as e:
    print("页面加载超时：", str(e))

soup = bs(driver.page_source, 'lxml')
driver.quit()

# 提取参展商名称
exhibitor_names = soup.find_all(class_="expofp-exhibitor-name")
for name in exhibitor_names:
    print(name.text.strip())

内容的提问来源于stack exchange，提问作者BQuist

爬虫爬取网站时遭遇旧版IE浏览器不兼容提示的问题求助

代码示例 #

输出内容 #

关键问题分析 #

方案一：直接抓取API数据 #

方案二：优化Selenium等待逻辑 #

代码示例

输出内容

关键问题分析

方案一：直接抓取API数据

方案二：优化Selenium等待逻辑