使用Scrapy XPath提取HTML中Gardıroplar文本失败问题排查

问题场景 #

需从指定HTML中提取“Gardıroplar”文本，但编写的Scrapy XPath返回空结果。

目标HTML结构 #

<ol class="nav align-items-center flex-nowrap text-nowrap overflow-auto hide-scrollbar">
<li>
  <a href="/">
    <svg class="icon-home m-0">
      <use
        xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-home"
      ></use>
    </svg>
  </a>
</li>
<li>
  <svg class="icon-arrow2 m-0">
    <use
      xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
    ></use>
  </svg>
  <a href="/mobilya/c/109">Mobilya</a>
</li>
<li>
  <svg class="icon-arrow1 m-0">
    <use
      xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
    ></use>
  </svg>
  <span class="top-breadcrumb">
    <a
      class="d-inline-flex align-items-center border pl-10 rounded-sm"
      href="/mobilya/gardiroplar/c/109011"
      data-toggle="dropdown"
      aria-expanded="false"
      >Gardıroplar<svg class="icon-arrow7 m-0 rotate-top">
        <use
          xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow7"
        ></use>
      </svg>
    </a>
    <ul class="dropdown-menu px-15 py-0 border-0 text-c2">
      <li class="border-bottom py-10">
        <a
          class="d-flex align-items-center justify-content-between pl-5 py-5 reverse font-weight-bold"
          href="/mobilya/gardiroplar/c/109011"
          >Gardıroplar</a
        >
      </li>

      <li class="px-10 border-bottom">
        <a
          class="d-flex align-items-center justify-content-between pl-5 py-5 reverse"
          href="/gardiroplar/kapakli-gardiroplar/c/109011002"
          >Kapaklı Gardıroplar<svg class="icon-arrow1 ml-5">
            <use
              xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
            ></use>
          </svg>
        </a>
      </li>
      <li class="px-10 border-bottom">
        <a
          class="d-flex align-items-center justify-content-between pl-5 py-5 reverse"
          href="/gardiroplar/surgulu-gardiroplar/c/109011003"
          >Sürgülü Gardıroplar<svg class="icon-arrow1 ml-5">
            <use
              xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
            ></use>
          </svg>
        </a>
      </li>
      <li class="px-10 border-bottom">
        <a
          class="d-flex align-items-center justify-content-between pl-5 py-5 reverse"
          href="/gardiroplar/bez-dolaplar/c/109011001"
          >Bez Dolaplar<svg class="icon-arrow1 ml-5">
            <use
              xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
            ></use>
          </svg>
        </a>
      </li>
    </ul>
  </span>
</li>
<li>
  <svg class="icon-arrow1 m-0">
    <use
      xlink:href="/_ui/responsive/theme-alpha/images/icons.svg#icon-arrow1"
    ></use>
  </svg>
  <a href="/gardiroplar/kapakli-gardiroplar/c/109011002">Kapaklı Gardıroplar</a>
</li>
</ol>

错误代码 #

response.xpath('//ol[@class="nav.align-items-center.flex-nowrap.text-nowrap.overflow-auto.hide-scrollbar.tab-title"]//li[svg[contains(@class,"icon-arrow2")]]/text()').getall()

错误原因分析 #

1. OL标签Class匹配错误：实际OL的class是空格分隔的多个类，你用点.连接，还额外加了不存在的tab-title类，导致无法定位到目标OL元素。
2. 路径指向错误：目标文本“Gardıroplar”在包含icon-arrow1的LI下的SPAN>A标签里，而你的XPath找的是包含icon-arrow2的LI的直接文本，完全偏离了目标位置。

正确的XPath写法 #

方式1：通过OL和SPAN的Class精准定位 #

# 提取面包屑中的Gardıroplar
response.xpath('//ol[@class="nav align-items-center flex-nowrap text-nowrap overflow-auto hide-scrollbar"]//span[@class="top-breadcrumb"]/a/text()').get()

方式2：通过A标签的HREF属性定位 #

# 提取所有指向该分类的A标签文本（会返回两个结果：面包屑和下拉菜单里的）
response.xpath('//a[@href="/mobilya/gardiroplar/c/109011"]/text()').getall()

方式3：直接匹配文本内容 #

# 提取所有文本为Gardıroplar的A标签内容
response.xpath('//a[text()="Gardıroplar"]/text()').get()

内容的提问来源于stack exchange，提问作者epope