使用Selenium爬取Compracerta商品图片返回空数组[]求助

核心问题排查 #

1. CSS选择器写法错误：你写的article.box-produto a.prod-info img.product_img_img__ac vtex-render-runtime-8-x-lazyload ls-is-cached lazyloaded中，类名间的空格代表「后代元素」，但这些类都属于同一个img标签，正确写法应该用.连接多个类（无需全写，选一个唯一标识类即可）。
2. 属性获取错误：图片链接存储在src或data-src（懒加载场景）属性里，而非text属性，直接取text必然返回空值。

修正后的代码 #

from pandas_datareader import data as pdr
import numpy as np
import pandas as pd
from datetime import datetime,timedelta
from bs4 import BeautifulSoup
import requests
import math
import re
from requests_html import HTMLSession, AsyncHTMLSession
from lxml import etree
import xlwt
import time
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
        

url_base='https://www.compracerta.com.br/celulares-e-smartphones?page=1'
executable_path = r'C:\Users\vinig\Downloads\chromedriver_win32\chromedriver.exe'
browser = webdriver.Chrome(executable_path=executable_path)
browser.get(url_base)
time.sleep(5)
WebDriverWait(browser,40).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
links = [my_elem.get_attribute("href") for my_elem in browser.find_elements(By.CSS_SELECTOR, "article.box-produto a.prod-info")]
titles = [my_elem.text for my_elem in browser.find_elements(By.CSS_SELECTOR, "article.box-produto a.prod-info h3 p")]
prices = [my_elem.text for my_elem in browser.find_elements(By.CSS_SELECTOR, "article.box-produto a.prod-info p span.por > span")]
# 修正图片逻辑
images = []
img_elements = browser.find_elements(By.CSS_SELECTOR, "article.box-produto a.prod-info img.product_img_img__ac")
for elem in img_elements:
    # 懒加载场景优先取data-src，否则用src
    img_url = elem.get_attribute("data-src") or elem.get_attribute("src")
    if img_url:
        images.append(img_url)
browser.quit()
print(links, titles, prices, images)

额外优化建议 #

• 替换固定等待time.sleep(5)为显式等待，确保图片加载完成后再抓取：

  WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "article.box-produto a.prod-info img.product_img_img__ac")))
  

• 若后续遇到反爬限制，可添加浏览器参数（如设置用户代理、禁用图片加载）降低被拦截概率。

内容的提问来源于stack exchange，提问作者Davi Riani