如何用Python+Selenium高效爬取网页底部图片的title属性？

Hey there! I totally get why you don’t want to rely on scrolling the entire window to the bottom—it’s a blunt approach and can be unreliable if page heights change or content loads dynamically. Let’s look at cleaner, more targeted solutions to grab those title attributes from the carousel images.

The Core Issue with Your Original scrollIntoView() Attempt #

Chances are your initial scrollIntoView() didn’t work because you tried to use it before the target elements were fully loaded in the DOM. Dynamic pages often delay rendering content until it’s needed, so we need to pair scrolling with explicit waits to ensure elements exist first.

Solution 1: Targeted Scrolling with Explicit Waits #

Instead of scrolling the whole window, we’ll wait for the carousel images to be present, then scroll each one into view (or just the carousel container itself) to trigger any lazy-loading logic. Here’s how to modify your script:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

URL = "https://www.99acres.com/supertech-cape-town-sector-74-noida-npxid-r922?sid=UiB8IFFTIHwgUyB8IzMxIyAgfCAxIHwgNyM0MyMgfCA4MjEyIHwjNSMgIHwg"
driver = webdriver.Chrome()
driver.get(URL)

# Wait up to 10 seconds for all carousel images to be present in the DOM
wait = WebDriverWait(driver, 10)
carousel_images = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "#carousel img")))

# Scroll each image into view to ensure any lazy-loaded metadata (like title) is loaded
for img in carousel_images:
    # Use smooth scrolling and center the element in the viewport
    driver.execute_script("arguments[0].scrollIntoView({block: 'center', behavior: 'smooth'});", img)
    # Optional: Wait for the image to be visible if needed (more reliable than time.sleep)
    # WebDriverWait(driver, 2).until(EC.visibility_of(img))
    print(img.get_attribute("title"))

driver.quit()

Why This Works: #

• Explicit waits: We don’t rely on arbitrary time.sleep() calls—we wait until the elements are actually present, which makes the script more stable across different load times.
• Targeted scrolling: We only scroll to the specific images we care about, not the entire page. The block: 'center' parameter ensures the element is fully in the viewport, which triggers any lazy-loading tied to visibility.
• Cleaner execution: No unnecessary scrolling makes the script faster and more efficient.

Solution 2: Scroll to the Carousel Container (Instead of Individual Images) #

If all images are inside the #carousel container, you can scroll to the container itself once, instead of scrolling each image:

# Wait for the carousel container to be present
carousel = wait.until(EC.presence_of_element_located((By.ID, "carousel")))
# Scroll the container into view
driver.execute_script("arguments[0].scrollIntoView({block: 'start', behavior: 'smooth'});", carousel)

# Now grab all images and their titles
carousel_images = driver.find_elements(By.CSS_SELECTOR, "#carousel img")
for img in carousel_images:
    print(img.get_attribute("title"))

This is even more efficient if the carousel loads all images once it’s in view.

Bonus Check: Do You Even Need to Scroll? #

Before scrolling, inspect the page’s HTML (using Chrome DevTools) to see if the title attributes are already present in the DOM for the images, even if they’re not visible. If they are, you can skip scrolling entirely and just wait for the elements to be present!

内容的提问来源于stack exchange，提问作者SIM