You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Python网页爬虫运行卡顿并出现两类错误的技术求助

问题描述

我正在完成学校项目,尝试用Python爬取网站数据,参考了Edureka的教程。以下是我的代码:

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd

driver = webdriver.Chrome("D:\COSC2625_Team_Blue\chromedriver")

products=[] #List to store name of the product
prices=[] #List to store price of the product
ratings=[] #List to store rating of the product
driver.get("""<a href="https://www.flipkart.com/laptops/">https://www.flipkart.com/laptops/</a>~buyback-guarantee-on-laptops-/pr?sid=6bo%2Cb5g&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;uniq""")

content = driver.page_source
soup = BeautifulSoup(content)
for a in soup.findAll('a',href=True, attrs={'class':'_31qSD5'}):
    name=a.find('div', attrs={'class':'_3wU53n'})
    price=a.find('div', attrs={'class':'_1vC4OE _2rQ-NK'})
    rating=a.find('div', attrs={'class':'hGSR34 _2beYZw'})
    products.append(name.text)
    prices.append(price.text)
    ratings.append(rating.text) 

df = pd.DataFrame({'Product Name':products,'Price':prices,'Rating':ratings}) 
df.to_csv('products.csv', index=False, encoding='utf-8')

运行代码后弹出空白浏览器,终端先出现这条错误:

PS D:\COSC2625_Team_Blue> [8556:2544:0809/132114.260:ERROR:device_event_log_impl.cc(214)] [13:21:14.260] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

查了下知道这个不影响运行,等了一会儿又出现第二条错误:

[4488:14404:0809/132312.207:ERROR:gpu_init.cc(486)] Passthrough is not supported, GL is disabled, ANGLE is 

之后程序彻底卡顿,等了一小时还是只有这两条错误,浏览器一直空白,请问这是什么原因?

排查与解决建议
  • 修正URL格式:driver.get()中的URL包含了HTML<a>标签,导致Chrome无法解析正确地址,直接卡在加载。改成纯链接:
    driver.get("https://www.flipkart.com/laptops/~buyback-guarantee-on-laptops-/pr?sid=6bo%2Cb5g&uniq")
    
  • 匹配ChromeDriver版本:确保ChromeDriver版本和本地Chrome浏览器版本完全一致,版本不兼容会引发启动异常或卡顿。
  • 添加页面等待:网站加载需要时间,直接获取page_source可能拿到空内容,添加显式等待确保元素加载完成:
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    # 最长等待10秒,直到目标元素出现
    WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "_31qSD5")))
    content = driver.page_source
    
  • 禁用GPU加速:GPU初始化错误可通过添加Chrome参数解决,避免浏览器卡顿:
    options = webdriver.ChromeOptions()
    options.add_argument("--disable-gpu")
    driver = webdriver.Chrome(r"D:\COSC2625_Team_Blue\chromedriver", options=options)
    
  • 修复路径转义:Windows路径中的反斜杠需转义,或使用原始字符串避免解析错误:
    driver = webdriver.Chrome(r"D:\COSC2625_Team_Blue\chromedriver")
    

内容的提问来源于stack exchange,提问作者Xiaolong Guo

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.08.22 19:36:16