如何用Python将深度嵌套XML转换为DataFrame？

需要将深度嵌套的BioC格式XML文件（johnny.xml）转换为包含注释信息的结构化Pandas DataFrame。

输入XML内容 #

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE collection SYSTEM "BioC.dtd">
<collection>
  <source/>
  <date/>
  <key/>
  <document>
    <id>2301222206</id>
    <infon key="tt_curatable">no</infon>
    <infon key="tt_version">1</infon>
    <infon key="tt_round">1</infon>
    <passage>
      <offset>0</offset>
      <text>Johnny likes pizza and chocolate, he lives in Italy with Emily.</text>
      <annotation id="1">
        <infon key="type">names</infon>
        <infon key="identifier">first_name</infon>
        <infon key="annotator">annotator_1</infon>
        <infon key="updated_at">2023-01-22T22:12:56Z</infon>
        <location offset="0" length="6"/>
        <text>Johnny</text>
      </annotation>
      <annotation id="3">
        <infon key="type">food</infon>
        <infon key="identifier"></infon>
        <infon key="annotator">annotator_2</infon>
        <infon key="updated_at">2023-01-22T22:13:51Z</infon>
        <location offset="13" length="19"/>
        <text>pizza and chocolate</text>
      </annotation>
      <annotation id="4">
        <infon key="type">location</infon>
        <infon key="identifier">europe</infon>
        <infon key="annotator">annotator_2</infon>
        <infon key="updated_at">2023-01-22T22:14:05Z</infon>
        <location offset="46" length="5"/>
        <text>Italy</text>
      </annotation>
      <annotation id="2">
        <infon key="type">names</infon>
        <infon key="identifier">first_name</infon>
        <infon key="annotator">annotator_1</infon>
        <infon key="updated_at">2023-01-22T22:13:08Z</infon>
        <location offset="57" length="5"/>
        <text>Emily</text>
      </annotation>
    </passage>
  </document>
</collection>

已尝试的失败方案 #

• 使用lxml基础遍历：仅提取表层节点，未深入到annotation层级，丢失注释详细信息

from lxml import objectify
root = objectify.parse('johnny.xml').getroot()
data=[]
for i in range(len(root.getchildren())):
    data.append([child.text for child in root.getchildren()[i].getchildren()])
df = pd.DataFrame(data)

• 递归函数方案：生成的DataFrame列数过多，不符合结构化需求
• pandas_read_xml库：生成47行数据，与预期输出不符
• BeautifulSoup方案：未得到目标结构化表格

使用lxml精准定位annotation节点，提取每个注释的所有相关字段（节点属性、子节点文本及子节点属性），整理为字典列表后转换为DataFrame：

from lxml import etree
import pandas as pd

# 解析XML文件
tree = etree.parse('johnny.xml')
root = tree.getroot()

# 存储所有注释数据的列表
annotations_data = []

# 遍历所有annotation节点
for annotation in root.findall('.//annotation'):
    # 初始化当前注释的字典
    ann_dict = {
        'annotation_id': annotation.get('id'),
        'text': annotation.find('text').text
    }
    
    # 提取所有infon节点的key和value
    for infon in annotation.findall('infon'):
        key = infon.get('key')
        value = infon.text if infon.text else ''
        ann_dict[key] = value
    
    # 提取location节点的offset和length属性
    location = annotation.find('location')
    ann_dict['offset'] = location.get('offset')
    ann_dict['length'] = location.get('length')
    
    # 添加到列表
    annotations_data.append(ann_dict)

# 转换为DataFrame
df = pd.DataFrame(annotations_data)
# 可选：调整列顺序，让结构更清晰
df = df[['annotation_id', 'type', 'identifier', 'annotator', 'updated_at', 'offset', 'length', 'text']]

print(df)

输出结果 #

内容的提问来源于stack exchange，提问作者learningtocode