Python3中soup.find无法获取全部<a>标签的问题咨询

尝试提取以下HTML代码中的所有<a>标签，但仅能获取到第一个href值。请问这是否是因为HTML代码为单行且无换行符导致的？

对应的HTML代码 #

<ul class="listing">

         <a name="anchor201"></a> 
    </ul>


        <div class="listing3">

            <ul class="listing"><li><div class="heading">Product 1</div></li><ul class="group" style="list-style-type:none;"><li><a href="/software/1229.html">  Documentation for Product 1 Series 101, Prod Release 1.2.29  </a></li><li><a href="/software/abc104.html"> Documentation for Product 1 Series 101, Prod Release 1.0.4 </a></li><li><a href="/software/b-rel1233.html">Documentation for Product 1 Series 101, Prod Release 1.2.33</a></li><li><a href="/software/notes-1232.html">Documentation for Product 1 Series 101, Prod Release 1.2.32</a></li><li><a href="/software/6528/notes-6528.html">Documentation for Product 1 Series 101, Prod Release 1.2.28</a></li><li><a href="/software/6526/notes-1226.html">Documentation for Product 1 Series 101, Prod Release 1.2.26</a></li><li><a href="/software/configure/guide/notes-1225.html">Documentation for Product 1 Series 101, Prod Release 1.2.25</a></li><li><a href="/software/configure/guide/notes-1142.html">Documentation for Product 1 Series 101, Prod Release 1.1.42 </a></li><li><a href="/software/notes-6132.html">Documentation for Product 1 Series 101, Prod Release 1.1.32 </a></li><li><a href="/software/abc123.html">Documentation for Product 1 Series 101, Prod Release 1.0.4.7</a></li></ul></ul>

        </div>

对应的Python代码 #

soup_full = BeautifulSoup(filehandle, "html.parser")
soup = soup_full.find('div', class_='listing3')
links = soup.find('ul', class_='listing')
links = links.find('div', class_='heading')
if links == None:
   links = soup_full.find('ul', class_='ungroup')    
   docs = links
else:
  heading = 1
while (links != None):
  if heading == 1:
     temp = links.string
     docs = links.nextSibling
     docs = soup.find('ul', class_='group')         
  docs2 = docs.find('a')      
  print ("test2", dosc2)
  try:
     geturl = docs2.get('href')
  except:
     print ("not working")
     continue

不是HTML单行无换行的问题，核心原因是你的代码逻辑存在多处问题：

• 使用docs.find('a')只会返回第一个匹配的<a>标签，而非所有符合条件的元素；
• 代码存在变量拼写错误：dosc2应为docs2；
• while循环未更新links变量，会陷入无限循环；
• 部分逻辑冗余，比如先获取links.nextSibling又立刻用soup.find('ul', class_='group')覆盖赋值。

修正后的代码可以直接定位到目标列表，一次性获取所有链接：

from bs4 import BeautifulSoup

soup_full = BeautifulSoup(filehandle, "html.parser")
# 定位到包含目标链接的div
listing3_div = soup_full.find('div', class_='listing3')
# 找到存放链接的ul容器
group_ul = listing3_div.find('ul', class_='group')
# 获取所有a标签
all_links = group_ul.find_all('a')

# 遍历输出每个href值
for link in all_links:
    href = link.get('href')
    print(href)

这段代码会输出所有<a>标签的href值，BeautifulSoup会自动解析HTML结构，不受代码是否换行的影响。

内容的提问来源于stack exchange，提问作者Sam