如何用Python结合Requests-html提取带in-match类的链接？

需求说明 #

我写了一段代码可以提取网页中的所有链接，但现在希望仅提取带有class="in-match"属性的<a>标签链接。以下是目标网页的HTML示例和当前代码，需要帮忙修改实现需求：

目标网页HTML示例 #

<tr><td class="h-text-left"><a href="/football/algeria/ligue-1/aso-chlef-cr-belouizdad/bNvFStWr/" class="in-match"><span>ASO Chlef</span> - <span>CR Belouizdad</span></a></td><td class="h-text-center"><a href="/football/algeria/ligue-1/aso-chlef-cr-belouizdad/bNvFStWr/">0:0</a></td><td class="table-main__odds" data-oid="6r9u7xv464x0xicel6" data-odd="3.24"></td><td class="table-main__odds colored" data-oid="6r9u7xv498x0x0"><span><span><span data-odd="2.89"></span></span></span></td><td class="table-main__odds" data-oid="6r9u7xv464x0xicel7" data-odd="2.30"></td><td class="h-text-right h-text-no-wrap">19.01.</td></tr>
<tr><td class="h-text-left"><a href="/football/algeria/ligue-1/constantine-es-ben-aknoun/lCXRPvG1/" class="in-match"><span>Constantine</span> - <span>Ben Aknoun</span></a></td><td class="h-text-center"><a href="/football/algeria/ligue-1/constantine-es-ben-aknoun/lCXRPvG1/">1:1</a></td><td class="table-main__odds" data-oid="6r9uaxv464x0xicelc" data-odd="1.19"></td><td class="table-main__odds colored" data-oid="6r9uaxv498x0x0"><span><span><span data-odd="5.57"></span></span></span></td><td class="table-main__odds" data-oid="6r9uaxv464x0xiceld" data-odd="14.73"></td><td class="h-text-right h-text-no-wrap">19.01.</td></tr>
<tr><td class="h-text-left"><a href="/football/algeria/ligue-1/el-bayadh-oran/fkYNQK0e/" class="in-match"><span>El Bayadh</span> - <span><strong>Oran</strong></span></a></td><td class="h-text-center"><a href="/football/algeria/ligue-1/el-bayadh-oran/fkYNQK0e/">0:1</a></td><td class="table-main__odds" data-oid="6r9u9xv464x0xicela" data-odd="1.64"></td><td class="table-main__odds" data-oid="6r9u9xv498x0x0" data-odd="3.30"></td><td class="table-main__odds colored" data-oid="6r9u9xv464x0xicelb"><span><span><span data-odd="5.76"></span></span></span></td><td class="h-text-right h-text-no-wrap">19.01.</td></tr>

当前代码 #

from requests_html import HTMLSession 

matchlink = 'https://www.betexplorer.com/football/algeria/ligue-1/results/'
 
session = HTMLSession()
 
r = session.get(matchlink)


links = r.html.links

for link in links:
    matchlinks = link
 
    session = HTMLSession()
 
    r = session.get('https://www.betexplorer.com' + matchlinks)
    r.html.render()
    print(link)

修改后的代码及说明 #

核心优化点 #

• 用CSS选择器精准定位目标标签，避免提取无关链接
• 复用同一个会话实例，减少资源消耗
• 直接提取目标标签的href属性，提升效率

修改后的代码 #

from requests_html import HTMLSession 

base_url = 'https://www.betexplorer.com'
matchlink = f'{base_url}/football/algeria/ligue-1/results/'
 
# 仅创建一次会话，复用连接
session = HTMLSession()
r = session.get(matchlink)

# 用CSS选择器筛选class为in-match的a标签
target_links = r.html.find('a.in-match')

for link in target_links:
    # 提取a标签的href属性
    match_href = link.attrs.get('href')
    if match_href:
        full_link = f'{base_url}{match_href}'
        # 保留原有的详情页请求逻辑
        detail_r = session.get(full_link)
        detail_r.html.render()
        print(full_link)

关键解释 #

• r.html.find('a.in-match')：通过CSS选择器直接定位所有符合条件的<a>元素，比遍历所有链接更高效精准
• link.attrs.get('href')：从选中的标签中提取链接地址，避免无效值
• 复用session：避免每次循环新建会话，提升请求速度并节省系统资源

内容的提问来源于stack exchange，提问作者Terra1