Python Selenium爬取网页表格问题：后四列返回空值

你的代码后4列持续返回空值，核心问题是使用绝对XPath全局定位元素，没有基于已获取的tr节点做范围限定的相对查找，同时冗余的重复代码、未跳过表头行、依赖固定时长休眠等问题，也会引发数据抓取异常。

关键修改点 #

• 基于当前tr元素使用相对XPath（以.开头），将查找范围限定在当前行内，避免全局查找的定位错误
• 优化异常处理，仅针对国家名称的两种嵌套结构做捕获，其余列统一简化处理
• 用显式等待替代固定time.sleep，确保表格完全加载后再抓取
• 跳过表头行，避免无效数据混入结果

修改后的完整代码 #

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import pandas as pd

url = 'https://www.adducation.info/general-knowledge-travel-and-transport/emergency-numbers/'
path= 'xxxxxxxxxx'  # 替换为你的ChromeDriver实际路径

service = Service(executable_path=path)
driver = webdriver.Chrome(service=service)
driver.get(url)

# 显式等待表格加载完成，确保所有行元素已渲染
wait = WebDriverWait(driver, 20)
containers = wait.until(EC.presence_of_all_elements_located((By.XPATH, '//tr')))

# 初始化数据存储列表
Country = []
Emergency = []
Police = []
Ambulance = []
Fire = []
Group = []
Calling_codes = []
Local_emergency_no = []

# 跳过表头行，从第2行开始遍历（索引1）
for tr in containers[1:]:
    # 处理国家名称的两种嵌套格式
    try:
        country = tr.find_element(By.XPATH, './td[1]/strong').text
    except:
        country = tr.find_element(By.XPATH, './td[1]/em/strong').text
    
    # 获取当前行所有单元格，统一提取文本
    tds = tr.find_elements(By.XPATH, './td')
    # 确保当前行有足够列数，避免索引越界
    if len(tds) >= 8:
        emergency = tds[1].text
        police = tds[2].text
        ambulance = tds[3].text
        fire = tds[4].text
        group = tds[5].text
        calling_code = tds[6].text
        local_no = tds[7].text
    else:
        # 列数不足时填充空值
        emergency = police = ambulance = fire = group = calling_code = local_no = ""
    
    # 将数据加入对应列表
    Country.append(country)
    Emergency.append(emergency)
    Police.append(police)
    Ambulance.append(ambulance)
    Fire.append(fire)
    Group.append(group)
    Calling_codes.append(calling_code)
    Local_emergency_no.append(local_no)

# 构建DataFrame
dict_ = {
    'Country': Country,
    'Emergency': Emergency, 
    'Police': Police, 
    'Ambulance': Ambulance, 
    'Fire': Fire, 
    'Continent': Group, 
    'Calling_codes': Calling_codes,
    'Local_emergency_no': Local_emergency_no
}

Emergency_DS = pd.DataFrame(dict_)
print(Emergency_DS)

driver.quit()  # 关闭浏览器释放资源

额外处理建议 #

如果修改后仍有部分列返回空值，可能是元素不在可视区域，可在获取tds前添加滚动操作：

driver.execute_script("arguments[0].scrollIntoView();", tr)

内容的提问来源于stack exchange，提问作者Mubaraq Onipede