如何对通过标签名提取的表格数据进行结构化整理？

我用Selenium写了一段通过标签名从表格提取数据的代码，但现在提取出的是零散的span文本，需要把它们整理成Contractor Name = FANCEE PLANTS LANDSCAPING这样的键值对形式。网站是动态的，没法用XPath、类名或ID，只能靠标签名定位。试过DataFrame但不知道怎么按前后span元素整理，求可行方案。

当前代码 #

data_coverage_dict = {}
row_id = 0
# Finally: Iterate through the ul elements
for tableElement in tableElements:
  trElements = tableElement.find_elements(By.TAG_NAME, 'tr')
  # print(f'aici este tr: {tableElement.text}')
  for tdelement in trElements:
      tdelements = tdelement.find_elements(By.TAG_NAME,'td')
      span_texts = []
      # print(f'aici este td: {tdelement.text}')

      for spanelement in tdelements:
        spanelements = spanelement.find_elements(By.TAG_NAME,'span')
        # data_coverage_table = []
        # for index in spanelements:
        #     row_data = [cell.text for cell in spanelements[index:index]]
        #     data_coverage_table.append(row_data)
        # print(f'data table:{data_coverage_table}')

        print(f'Span element: {spanelement.text}')

当前输出 #

Span element: Contractor Name
Span element: FANCEE PLANTS LANDSCAPING
Span element: DBA
Span element: License
Span element: 439423
Span element: City
Span element: RIVERSIDE
Span element: Status
Span element: Cancelled
Span element: Contractor Name
Span element: FANCHBILT INCORPORATED
Span element: Name Type
Span element: DBA
Span element: License
Span element: 520066
Span element: City
Span element: LAGUNA WOODS
Span element: Status
Span element: Cancelled

目标网站表格结构示例 #

<table>Info</table>
 <tr>info tr</tr>
  <td>First element(in my case:Contract Name)</td>
  <td>Second element(in my case:FANCEE PLANTS LANDSCAPING)</td>

核心思路是成对处理相邻的span文本，从输出和表格结构能看出，每个键（如Contractor Name）和对应值（如FANCEE PLANTS LANDSCAPING）是连续出现的，且每次出现Contractor Name代表新的承包商条目开始。

方法1：收集所有span文本后成对分组 #

先把所有有效span文本存入列表，再按顺序两两配对，遇到Contractor Name就新建字典存储当前承包商信息，最后汇总所有条目。

修改后的代码：

from selenium.webdriver.common.by import By

# 存储所有承包商信息的列表
contractors = []
# 临时存储当前承包商的键值对
current_contractor = {}
# 临时记录上一个span文本（作为键）
last_key = None

for tableElement in tableElements:
    trElements = tableElement.find_elements(By.TAG_NAME, 'tr')
    for tr in trElements:
        tds = tr.find_elements(By.TAG_NAME, 'td')
        for td in tds:
            spans = td.find_elements(By.TAG_NAME, 'span')
            for span in spans:
                text = span.text.strip()
                if not text:
                    continue  # 跳过空文本
                
                if text == 'Contractor Name':
                    # 新承包商开始，先保存之前的条目
                    if current_contractor:
                        contractors.append(current_contractor)
                    current_contractor = {}
                    last_key = text
                else:
                    if last_key:
                        # 上一个文本作为键，当前文本作为值
                        current_contractor[last_key] = text
                        last_key = None
                    else:
                        # 当前文本作为下一个值的键
                        last_key = text

# 保存最后一个承包商条目
if current_contractor:
    contractors.append(current_contractor)

# 打印整理后的结果
for idx, contractor in enumerate(contractors, 1):
    print(f"承包商 {idx}:")
    for key, value in contractor.items():
        print(f"{key} = {value}")
    print("---")

方法2：遍历tr/td时直接提取键值对 #

根据表格结构，每个<tr>包含两个<td>，分别对应键和值，可以直接在遍历tr时提取对应内容，效率更高。

修改后的代码：

from selenium.webdriver.common.by import By

contractors = []
current_contractor = {}

for tableElement in tableElements:
    trElements = tableElement.find_elements(By.TAG_NAME, 'tr')
    for tr in trElements:
        tds = tr.find_elements(By.TAG_NAME, 'td')
        # 确保当前tr包含两个td（键和值）
        if len(tds) == 2:
            # 提取键和对应的值
            key = tds[0].find_element(By.TAG_NAME, 'span').text.strip()
            value = tds[1].find_element(By.TAG_NAME, 'span').text.strip()
            
            if key == 'Contractor Name':
                # 新承包商开始，保存之前的条目
                if current_contractor:
                    contractors.append(current_contractor)
                current_contractor = {key: value}
            else:
                current_contractor[key] = value

# 保存最后一个承包商条目
if current_contractor:
    contractors.append(current_contractor)

# 打印整理后的结果
for contractor in contractors:
    for k, v in contractor.items():
        print(f"{k} = {v}")
    print("---")

转成DataFrame（可选） #

如果需要用DataFrame存储，直接将承包商列表传入即可：

import pandas as pd

df = pd.DataFrame(contractors)
print(df)

内容的提问来源于stack exchange，提问作者Georgian Costea