网页爬取遇AttributeError：'NoneType'对象无'attrs'属性如何解决？

问题场景 #

使用Python的requests和BeautifulSoup库爬取Jumia埃及站的笔记本电脑数据时，尝试获取商品链接列表触发以下错误：

原代码 #

import requests
import bs4
import csv
from itertools import zip_longest

laptop = []
laptops_price = []
links = []

url = "https://www.jumia.com.eg/ar/catalog/?q=%D9%84%D8%A7%D8%A8%D8%AA%D9%88%D8%A8"
page = requests.get("https://www.jumia.com.eg/ar/catalog/?q=%D9%84%D8%A7%D8%A8%D8%AA%D9%88%D8%A8")
bs = bs4.BeautifulSoup(page.content, 'html.parser')
laptops = bs.find_all('h3')
laptops_prices = bs.find_all("div", {"class": "prc"})
for l in range(len(laptops)):
    laptop.append(laptops[l].text)
    links.append(laptops[l].find("a", {"class" : "core"}).attrs['href'])
    laptops_price.append(laptops_prices[l].text)


laptops_list = [laptop, laptops_price, links]
exported = zip_longest(*laptops_list)
with open(r"C:\Users\Administrator\Desktop\jumiawep.csv", "w", encoding="utf-8") as jumialaptops:
    write = csv.writer(jumialaptops)
    write.writerow(["Laptop", "Price", "Links"])
    write.writerows(exported)

报错信息 #

Traceback (most recent call last):
  File "C:\Users\Administrator\PycharmProjects\pythonProject\main.py", line 17, in <module>
    links.append(laptops[l].find("a").attrs['href'])
AttributeError: 'NoneType' object has no attribute 'attrs'

错误原因 #

1. 标签匹配失败：并非所有<h3>标签内部都存在带core类的<a>标签，当find()方法找不到目标元素时会返回None，此时访问None.attrs必然触发属性错误。
2. 条目数量不匹配：单独提取的<h3>列表和价格<div class="prc">列表长度可能不一致，直接按索引循环会导致潜在的索引越界问题。

修复方案 #

直接定位包含完整商品信息的容器标签，对每个元素的查找结果增加空值判断，确保数据提取的稳定性：

修改后代码 #

import requests
import bs4
import csv
from itertools import zip_longest

laptop = []
laptops_price = []
links = []

url = "https://www.jumia.com.eg/ar/catalog/?q=%D9%84%D8%A7%D8%A8%D8%AA%D9%88%D8%A8"
page = requests.get(url)
bs = bs4.BeautifulSoup(page.content, 'html.parser')
# 定位单个商品的完整容器，确保信息完整性
product_containers = bs.find_all('article', class_='prd _fb col c-prd')

for container in product_containers:
    # 提取商品名称
    name_elem = container.find('h3', class_='name')
    if name_elem:
        laptop.append(name_elem.get_text(strip=True))
    else:
        laptop.append("无名称")
    
    # 提取并拼接完整商品链接
    link_elem = container.find('a', class_='core')
    if link_elem and 'href' in link_elem.attrs:
        full_link = f"https://www.jumia.com.eg{link_elem['href']}"
        links.append(full_link)
    else:
        links.append("无链接")
    
    # 提取商品价格
    price_elem = container.find('div', class_='prc')
    if price_elem:
        laptops_price.append(price_elem.get_text(strip=True))
    else:
        laptops_price.append("无价格")

# 导出CSV
laptops_list = [laptop, laptops_price, links]
exported = zip_longest(*laptops_list)
with open(r"C:\Users\Administrator\Desktop\jumiawep.csv", "w", encoding="utf-8", newline='') as jumialaptops:
    writer = csv.writer(jumialaptops)
    writer.writerow(["Laptop", "Price", "Links"])
    writer.writerows(exported)

关键修改说明 #

• 定位商品容器：直接抓取<article class="prd _fb col c-prd">标签，每个容器对应一件商品，避免单独提取标签导致的信息不匹配。
• 空值判断：对每个元素的查找结果做校验，若找不到则填充默认值，避免报错。
• 完整链接拼接：原链接是相对路径，拼接成完整URL方便直接访问。
• 优化CSV导出：添加newline=''参数，避免生成的CSV出现多余空行。

内容的提问来源于stack exchange，提问作者Mohammed Nady