如何用Python动态提取页面树中的最内层嵌套链接？

Approach #

Use BeautifulSoup to parse HTML, locate the section with your target title, then filter out links that contain child links—these remaining links are the innermost ones in their respective branches.

Sample HTML #

<div class="page-tree">
  <div class="topic">
    <h2>Main Topic</h2>
    <div class="subtopic-group">
      <div class="subtopic">
        <a href="/sub1">Subtopic 1</a>
      </div>
      <div class="subtopic">
        <a href="/sub2">Subtopic 2</a>
        <div class="sub-subtopic">
          <p>Additional content</p>
        </div>
      </div>
      <div class="subtopic">
        <a href="/parent">Parent Topic</a>
        <div class="sub-subtopic">
          <a href="/child">Child Topic</a>
        </div>
      </div>
    </div>
  </div>
</div>

Python Script #

from bs4 import BeautifulSoup

def get_innermost_links(html, target_title):
    soup = BeautifulSoup(html, 'html.parser')
    
    # Locate the section with the target title
    heading = soup.find(['h1', 'h2', 'h3', 'h4'], string=lambda text: target_title in text.strip())
    if not heading:
        return []
    
    # Get all links under this section
    section = heading.find_next_sibling()
    all_links = section.find_all('a')
    
    # Filter for links with no child links (innermost)
    innermost = [link for link in all_links if not link.find('a')]
    
    return innermost

# Example usage
if __name__ == "__main__":
    sample_html = """
    <div class="page-tree">
      <div class="topic">
        <h2>Main Topic</h2>
        <div class="subtopic-group">
          <div class="subtopic">
            <a href="/sub1">Subtopic 1</a>
          </div>
          <div class="subtopic">
            <a href="/sub2">Subtopic 2</a>
            <div class="sub-subtopic">
              <p>Additional content</p>
            </div>
          </div>
          <div class="subtopic">
            <a href="/parent">Parent Topic</a>
            <div class="sub-subtopic">
              <a href="/child">Child Topic</a>
            </div>
          </div>
        </div>
      </div>
    </div>
    """
    
    result = get_innermost_links(sample_html, "Main Topic")
    for link in result:
        print(f"Link Text: {link.text}, URL: {link['href']}")

Output #

Link Text: Subtopic 1, URL: /sub1
Link Text: Subtopic 2, URL: /sub2
Link Text: Child Topic, URL: /child

This script adapts to any nesting level because it checks each link for child links—regardless of how deep a branch goes, it picks the last (innermost) link in that path.

内容的提问来源于stack exchange，提问作者Rachel Negi