Selenium(Python)网页爬取XPATH定位失败，求解决方案

我使用Python的Selenium爬取某网站（HTML已匿名处理），但元素定位获取数据的部分无法正常工作，需要提取INFO 1、INFO 2、INFO 3、TEXT 1这几项信息。尝试过以下XPATH代码但未能成功：

INFO_1= driver.find_elements(By.XPATH, "/html/body/div[1]/div/div/div[2]/section[2]/div[2]/div[1]/div/a/div[3]/div/span[1]")

对应的目标HTML代码如下：

<a href=" anonymized href " title=" **TITLE** " class="suggestion-link">
   <div>
<a href="#" style="display: inherit;"><span class="fav"></span></a></div> 
<div class="image-wrap">
   <!----> <!----> <!----> <!----> <!----> <!----> 
   <div class="carousel">
      <span id="carousel_prev_8iikolcpmwi" style="display: none;"></span> 
      <div id="carousel_o2u8ikkx7nl" class="owl-carousel owl-theme owl-loaded owl-drag">
         <div class="owl-stage-outer">
            <div class="owl-stage" style="transform: translate3d(0px, 0px, 0px); transition: all 0s ease 0s; width: 970px;">
               <div class="owl-item active" style="width: 323.242px;">
                  <div class="item">
                     <div class="filigrane"><img src="/images/filigrane.png"></div>
                     <div class="loaded">< anonymised.jpg" alt=" **TITLE 2**" class="img"> </div>
                  </div>
               </div>
               <div class="owl-item" style="width: 323.242px;">
                  <div class="item">
                     <div class="filigrane"><img src="/images/filigrane.png"></div>
                     <div class="loaded"><img src=" anonymized.jpg" alt=" **TITLE** " class="img"> </div>
                  </div>
               </div>
               <div class="owl-item" style="width: 323.242px;">
                  <div class="item">
                     <div class="filigrane"><img src="/images/filigrane.png"></div>
                     <div class="loaded"><img src=" anonymised.jpg" alt=" **TITLE** " class="img"> </div>
                  </div>
               </div>
            </div>
         </div>
         <div class="owl-nav disabled">
            <div class="owl-prev">next</div>
            <div class="owl-next">prev</div>
         </div>
         <div class="owl-dots"><button role="button" class="owl-dot active"><span></span></button><button role="button" class="owl-dot"><span></span></button><button role="button" class="owl-dot"><span></span></button></div>
      </div>
      <span id="carousel_next_d6crxou5xy"></span>
   </div>
</div>
<div class="content-wrap">
   <div class="card-top">
      <span class="card-left uppercase"> **INFO 1**</span> 
      <span class="card-right">
         <!----> <!---->
         **INFO 2**
         <!---->
      </span>
   </div>
   <h3 class="title-wrap">**INFO 3**</h3>
   <p class=""><span class="moreup"></span> **TEXT 1**.</p>
</div>
</a>

你使用的绝对XPATH过于依赖页面层级结构，一旦页面布局微调就会失效，建议改用基于元素属性的相对定位，稳定性更强：

各信息提取代码 #

• 提取INFO 1：

info_1 = driver.find_element(By.XPATH, "//a[@class='suggestion-link']//span[@class='card-left uppercase']").text.strip()

• 提取INFO 2：

info_2 = driver.find_element(By.XPATH, "//a[@class='suggestion-link']//span[@class='card-right']").text.strip()

• 提取INFO 3：

info_3 = driver.find_element(By.XPATH, "//a[@class='suggestion-link']//h3[@class='title-wrap']").text.strip()

• 提取TEXT 1：

text_1 = driver.find_element(By.XPATH, "//a[@class='suggestion-link']//p[span[@class='moreup']]").text.strip()

注意事项 #

1. 若页面存在多个具有suggestion-link类的<a>标签，改用find_elements方法获取所有元素，再循环遍历处理每个元素的信息。
2. 确保元素完全加载后再进行定位，可使用WebDriverWait等待元素可见，避免因页面未加载完成导致的定位失败：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 示例：等待INFO 1元素加载完成
info_1_element = WebDriverWait(driver, 10).until(
    EC.visibility_of_element_located((By.XPATH, "//a[@class='suggestion-link']//span[@class='card-left uppercase']"))
)
info_1 = info_1_element.text.strip()

内容的提问来源于stack exchange，提问作者jacopo salmistrari