使用BeautifulSoup提取网页社交媒体链接时报KeyError:'href'如何解决？

目标提取页面 https://keithgalli.github.io/web-scraping/webpage.html 的所有社交媒体链接，初始编写代码如下：

import requests 
from bs4 import BeautifulSoup as bs

r = requests.get('https://keithgalli.github.io/web-scraping/webpage.html')
soup = bs(r.content)

links = soup.find_all('a', {'class':'socials'})

actual_links = [link['href'] for link in links]

运行抛出错误：

KeyError: 'href'

经排查，错误原因是该站点的socials类绑定在社交媒体链接的外层<ul>容器上，而非单个<a>标签上。原代码查询返回的列表元素为完整的<ul class="socials">节点，并非单个可直接提取href属性的<a>标签，迭代取值时触发报错。

先定位到socials类的容器节点，再在容器内部查询所有<a>标签后提取href属性即可，修复后代码如下：

import requests 
from bs4 import BeautifulSoup as bs

r = requests.get('https://keithgalli.github.io/web-scraping/webpage.html')
soup = bs(r.content)

# 定位社交媒体链接外层容器
socials_ul = soup.find('ul', {'class': 'socials'})
# 容器内查找所有a标签
social_links = socials_ul.find_all('a')
# 批量提取href属性
actual_links = [link['href'] for link in social_links]

如果站点存在多个socials类容器，可使用以下兼容写法汇总所有链接：

import requests 
from bs4 import BeautifulSoup as bs

r = requests.get('https://keithgalli.github.io/web-scraping/webpage.html')
soup = bs(r.content)

actual_links = []
socials_containers = soup.find_all('ul', {'class': 'socials'})
for container in socials_containers:
    links = container.find_all('a')
    actual_links.extend([link['href'] for link in links])

内容的提问来源于stack exchange，提问作者swordlordswamplord