如何用Selenium捕获无限滚动网站的div元素？Instagram批量点赞问题

Hey there, let's tackle this Instagram scrolling/element collection issue you're facing!

The core problem here is Instagram's virtual scrolling implementation — when you scroll past the first few rows of posts, the platform removes older post divs from the DOM to save memory. That means the WebElement objects you stored in your ArrayList immediately become stale (invalid) once those divs disappear, making them impossible to interact with later.

Instead of trying to store the WebElements directly, here's a reliable workaround:

Step 1: Collect all post URLs first (not WebElements) #

Instead of grabbing the divs themselves, extract the unique URL for each post and store those in a set (to avoid duplicates). URLs don't become stale like DOM elements, so you can safely reuse them later.

Step 2: Scroll incrementally until no new posts load #

You'll need to simulate scrolling to the bottom of the page, wait for new posts to load, and repeat until you've captured every post URL.

Step 3: Iterate through the URLs to like each post #

Once you have all post links, visit each one individually, locate the like button, and perform the action. This avoids interacting with stale elements entirely.

Here's a concrete implementation:

import org.openqa.selenium.*;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class InstagramLiker {
    private WebDriver driver;
    private Set<String> postLinks = new HashSet<>();
    private JavascriptExecutor jsExecutor;

    public InstagramLiker(WebDriver driver) {
        this.driver = driver;
        this.jsExecutor = (JavascriptExecutor) driver;
    }

    // Collect all post URLs from the target profile
    public void collectAllPostLinks() {
        long lastPageHeight = (long) jsExecutor.executeScript("return document.body.scrollHeight");

        while (true) {
            // Grab all visible post links on the current view
            List<WebElement> currentPostElements = driver.findElements(
                By.cssSelector("#react-root > section > main > div > div._2z6nI > article > div > div > div a")
            );
            
            // Extract and store each post's URL
            for (WebElement postElement : currentPostElements) {
                String postUrl = postElement.getAttribute("href");
                if (postUrl != null && !postUrl.isEmpty()) {
                    postLinks.add(postUrl);
                }
            }

            // Scroll to the bottom of the page
            jsExecutor.executeScript("window.scrollTo(0, document.body.scrollHeight);");

            // Wait for new posts to load (replace with explicit wait for reliability)
            try {
                Thread.sleep(2000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }

            // Check if we've reached the end of the profile (no new content)
            long newPageHeight = (long) jsExecutor.executeScript("return document.body.scrollHeight");
            if (newPageHeight == lastPageHeight) {
                break;
            }
            lastPageHeight = newPageHeight;
        }
        System.out.println("Collected " + postLinks.size() + " post links");
    }

    // Like every post using the collected URLs
    public void likeAllPosts() {
        WebDriverWait wait = new WebDriverWait(driver, 10);

        for (String postUrl : postLinks) {
            driver.get(postUrl);

            try {
                // Wait for the like button to be clickable
                WebElement likeButton = wait.until(
                    ExpectedConditions.elementToBeClickable(By.cssSelector("section.ltpMr.Slqrh span.fr66n button"))
                );

                // Only click if the post isn't already liked
                String buttonLabel = likeButton.getAttribute("aria-label");
                if ("Like".equals(buttonLabel)) {
                    likeButton.click();
                    System.out.println("Liked post: " + postUrl);
                } else {
                    System.out.println("Post already liked: " + postUrl);
                }

                // Add a small delay to mimic human behavior (avoid Instagram limits)
                Thread.sleep(1000);
            } catch (Exception e) {
                System.out.println("Failed to like post: " + postUrl);
                e.printStackTrace();
            }
        }
    }
}

Key Notes: #

• Avoid Thread.sleep where possible: Replace fixed sleeps with WebDriverWait (like the one waiting for the like button) to make the code more reliable across different network speeds.
• Watch for Instagram's anti-bot measures: Frequent automated actions can trigger rate limits or account restrictions. Add longer delays between actions if needed.
• Update CSS selectors: Instagram changes its frontend code occasionally, so double-check that your selectors still target the right elements if the code stops working.

内容的提问来源于stack exchange，提问作者Mkay