解析含韩文XML遇lxml.etree.XMLSyntaxError问题求助

解决XML解析中未完成的CDATA节错误 #

1. 定位问题根源 #

这个错误并非韩文本身导致，而是API返回的XML存在未正确闭合的CDATA节（缺失]]>标记），或者编码识别错误导致解析器误判CDATA边界。先通过以下代码排查：

import requests
import re

url = "https://api.lever.co/v0/postings/matchgroup?mode=xml"
r = requests.get(url)
# 确认返回编码
print("返回编码:", r.encoding)
# 解码为字符串检查CDATA
content_str = r.content.decode('utf-8')
# 查找未闭合的CDATA
unclosed_cdata = re.findall(r'<!\[CDATA\[.*?(?<!\]\]>)', content_str, re.DOTALL)
if unclosed_cdata:
    print("发现未闭合CDATA:", unclosed_cdata)

2. 预处理修复CDATA #

如果确认是CDATA未闭合，解析前手动修补：

import lxml.etree
import requests
import re

url = "https://api.lever.co/v0/postings/matchgroup?mode=xml"
r = requests.get(url)
content_str = r.content.decode('utf-8')

# 修补所有未闭合的CDATA节，在末尾补全]]>
content_fixed = re.sub(
    r'<!\[CDATA\[([^\]]*(?:\](?!\])[^\]]*)*)$',
    r'<!\[CDATA[\1]]>',
    content_str,
    flags=re.DOTALL
)

# 用修复后的内容解析XML
parser = lxml.etree.XMLParser(recover=False, encoding='utf-8')
tree = lxml.etree.fromstring(content_fixed.encode('utf-8'), parser)

3. 强制指定UTF-8编码解析 #

若问题源于编码识别错误，强制使用UTF-8解码后再解析：

import lxml.etree
import requests

url = "https://api.lever.co/v0/postings/matchgroup?mode=xml"
r = requests.get(url)
# 强制按UTF-8解码后重新编码，避免编码混乱
content = r.content.decode('utf-8').encode('utf-8')
parser = lxml.etree.XMLParser(recover=False, encoding='utf-8')
tree = lxml.etree.fromstring(content, parser)

4. 降级为HTML解析（备选方案） #

如果以上方法无效，可先以HTML解析器处理（对格式要求更宽松），再转换为XML结构：

from lxml import html, etree
import requests

url = "https://api.lever.co/v0/postings/matchgroup?mode=xml"
r = requests.get(url)
# 用HTML解析器加载内容
html_tree = html.fromstring(r.content)
# 转换为XML格式字符串
xml_str = etree.tostring(html_tree, encoding='utf-8').decode('utf-8')
# 解析XML
parser = etree.XMLParser(recover=False)
tree = etree.fromstring(xml_str.encode('utf-8'), parser)

内容的提问来源于stack exchange，提问作者Asher Ross