Python爬虫获取元素时出现重复计数与多余分隔符问题

Hey there, let's work through these issues one by one—since you're new to Python and Selenium, these are super common pitfalls, so you're not alone here!

1. Why is your count double the actual element number? #

This usually happens because your element locator is matching duplicate or hidden elements on the page. For example:

• Your selector might be picking up both visible and hidden elements (like mobile/desktop versions of the same component, where one is hidden with CSS).
• You're using a too-broad locator (e.g., find_elements_by_tag_name("p")) that matches parent and child elements that both count as separate entries.

Fix: #

• Refine your locator: Use a more specific CSS selector or XPath that targets only the visible, intended elements. For example, if your items are in a list with class data-item, use By.CSS_SELECTOR, ".data-item" instead of a generic tag.
• Filter out hidden elements: Add a check to only count elements that are visible:

  for e in elements:
      if e.is_displayed():  # Skip hidden elements
          # Your code here
  

• Verify in dev tools: Open your browser's DevTools (F12), go to the Elements tab, and run document.querySelectorAll("your-selector") in the console. This will show all elements matching your locator—you might spot duplicates you didn't notice!

2. Extra blank lines or double commas #

This ties into the duplicate/hidden element issue: some of the elements you're looping through have empty text content. When you append a newline or comma to an empty string, you end up with extra separators.

Fix: #

Only add text to your list if it's not empty (after stripping whitespace):

elelist = []
count = 0

elements = driver.find_elements(By.CSS_SELECTOR, "your-specific-selector")

for e in elements:
    item_text = e.text.strip()  # Remove leading/trailing whitespace
    if item_text:  # Only process non-empty text
        elelist.append(item_text)
        count += 1

# Join with commas (no doubles!)
comma_separated = ", ".join(elelist)
# Or join with newlines (no blank lines!)
newline_separated = "\n".join(elelist)

3. elements.count returns a method object instead of a number #

Ah, that's a Python list quirk! count() is a method for lists that lets you count how many times a specific item appears in the list (e.g., elements.count(some_element)). To get the total number of elements, you need to use len(elements) instead.

Example: #

total_elements = len(elements)
print(f"Total elements found: {total_elements}")

Quick Recap of Steps to Debug #

• Double-check your locator in DevTools to ensure it only targets the elements you want.
• Filter out hidden/empty-text elements in your loop.
• Use len() to get the total number of elements, not count().

内容的提问来源于stack exchange，提问作者gritts