如何爬取特定营养数据并导入Pandas DataFrame？解决无TH表格列值区分

1. 核心思路 #

该表格的结构特征是每一行包含两个class为column的<td>标签：第一个是营养项名称（表头），第二个是对应数值。我们可以通过按步长遍历的方式区分这两类标签，再筛选目标宏量营养素后存入DataFrame。

2. 代码实现（静态页面爬取） #

import requests
from bs4 import BeautifulSoup
import pandas as pd

# 请求页面（添加UA避免被拦截）
url = "https://www.jumbo.com/producten/jumbo-scharrelkip-filet-800g-515026BAK"
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, "html.parser")

# 定位营养值表格
nutrition_table = soup.find("table", class_=lambda c: c and "nutrition" in c)
if not nutrition_table:
    nutrition_section = soup.find(text=lambda t: t and "Voedingswaarden" in t)
    nutrition_table = nutrition_section.find_next("table")

# 提取所有class为column的td标签
td_elements = nutrition_table.find_all("td", class_="column")

# 分组提取营养项和对应值
nutrition_dict = {}
for idx in range(0, len(td_elements), 2):
    nutrient_name = td_elements[idx].get_text(strip=True)
    nutrient_value = td_elements[idx+1].get_text(strip=True)
    nutrition_dict[nutrient_name] = nutrient_value

# 筛选红色标记的宏量营养素
target_nutrients = ["Energie", "Vet", "Verzadigd vet", "Koolhydraten", "Suikers", "Eiwitten", "Zout"]
filtered_data = {k: v for k, v in nutrition_dict.items() if k in target_nutrients}

# 转换为DataFrame
df = pd.DataFrame([filtered_data])
print(df)

3. 动态页面兼容方案（若静态请求失败） #

如果页面是动态加载的，改用Selenium获取渲染后的内容：

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
import pandas as pd

driver = webdriver.Chrome()
driver.get("https://www.jumbo.com/producten/jumbo-scharrelkip-filet-800g-515026BAK")
time.sleep(2)  # 等待页面渲染完成

# 定位营养表格
nutrition_table = driver.find_element(By.XPATH, "//table[.//*[contains(text(), 'Voedingswaarden')]]")
td_elements = nutrition_table.find_elements(By.CLASS_NAME, "column")

# 分组提取数据
nutrition_dict = {}
for idx in range(0, len(td_elements), 2):
    nutrient_name = td_elements[idx].text.strip()
    nutrient_value = td_elements[idx+1].text.strip()
    nutrition_dict[nutrient_name] = nutrient_value

# 筛选目标营养素并转DataFrame
target_nutrients = ["Energie", "Vet", "Verzadigd vet", "Koolhydraten", "Suikers", "Eiwitten", "Zout"]
filtered_data = {k: v for k, v in nutrition_dict.items() if k in target_nutrients}
df = pd.DataFrame([filtered_data])

driver.quit()
print(df)

关键说明 #

• 依赖表格的行结构规律：每两个<td>为一组，分别对应营养项和数值，通过步长为2的循环实现区分。
• 目标营养素列表可根据红色标记内容调整，确保只提取需要的宏量营养素数据。

内容的提问来源于stack exchange，提问作者MLP99