如何解析嵌套XML并提取属性与标签文本，高效生成目标DataFrame？

需求说明 #

解析嵌套XML，提取指定属性与标签文本并转换为DataFrame，要求排除<visits>标签下的<name>字段，仅保留<offer>层级的<name>字段，同时避免为每个字段编写单独的提取循环。

原始XML结构 #

<?xml version="1.0" encoding="UTF-8" ?>
<main_heading timestamp="20220113">
<details>
    <offer id="11" new_id="12">
        <level>1&amp;1</level>
        <typ>Green</typ>
        <name>Alpha</name>
        <visits>
            <name>DONT INCLUDE</name>
        </visits>
    </offer>
    <offer id="12" new_id="31">
        <level>1&amp;1</level>
        <typ>Yellow</typ>
        <name>Beta</name>
        <visits>
            <name>DONT INCLUDE</name>
        </visits>
    </offer>
</details>
</main_heading>

预期DataFrame输出 #

timestamp   id     new_id   level      name
20220113    11     12       1&1    Alpha
20220113    12     31       1&1    Beta

优化实现代码 #

from bs4 import BeautifulSoup
import pandas as pd

# 替换为你的XML内容字符串
xml_content = """<?xml version="1.0" encoding="UTF-8" ?>
<main_heading timestamp="20220113">
<details>
    <offer id="11" new_id="12">
        <level>1&amp;1</level>
        <typ>Green</typ>
        <name>Alpha</name>
        <visits>
            <name>DONT INCLUDE</name>
        </visits>
    </offer>
    <offer id="12" new_id="31">
        <level>1&amp;1</level>
        <typ>Yellow</typ>
        <name>Beta</name>
        <visits>
            <name>DONT INCLUDE</name>
        </visits>
    </offer>
</details>
</main_heading>"""

# 解析XML
soup = BeautifulSoup(xml_content, 'xml')

# 获取全局timestamp
timestamp = soup.find('main_heading').get('timestamp')

# 批量提取每个offer的信息
data = []
for offer in soup.find_all('offer'):
    # 初始化当前offer的数据字典，先加入timestamp和属性
    offer_info = {
        'timestamp': timestamp,
        'id': offer.get('id'),
        'new_id': offer.get('new_id')
    }
    # 遍历offer的直接子节点，提取有效标签文本
    for child in offer.children:
        # 跳过空白节点和visits标签
        if child.name and child.name != 'visits':
            offer_info[child.name] = child.text.strip()
    data.append(offer_info)

# 转换为DataFrame并调整列顺序
df = pd.DataFrame(data)[['timestamp', 'id', 'new_id', 'level', 'name']]
print(df)

方案优势 #

• 无需为level、typ、name等字段单独编写循环，通过遍历<offer>的直接子节点实现批量提取
• 直接跳过<visits>标签，自然排除了其下的<name>字段，无需额外过滤逻辑
• 代码扩展性强，若后续<offer>下新增其他标签，无需修改提取逻辑

内容的提问来源于stack exchange，提问作者x89