基于Saxonche的Python代码持续内存泄漏问题排查求助

环境 #

• Python: 3.11
• Saxonche: 12.4.2

问题描述 #

我的网站持续占用越来越多内存，直至服务器内存耗尽崩溃。已将问题代码隔离为如下测试脚本，该脚本内存占用速率约为0.5MB/秒，无平台期，内存会持续增长数小时直至崩溃。

import gc
from time import sleep

from saxonche import PySaxonProcessor


xml_str = """
<root>
    <stuff>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ac auctor ex. Nunc in tincidunt urna. Sed tincidunt eros lacus, sed pulvinar sem venenatis et. Donec euismod orci quis pellentesque sagittis. Donec at tortor in dui mattis facilisis. Pellentesque vel varius lectus. Nunc sed gravida risus, ac finibus elit. Etiam sollicitudin nunc a velit efficitur molestie in ac lectus. Donec vulputate orci odio, sit amet hendrerit odio rhoncus commodo.</stuff>
    <stuff>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ac auctor ex. Nunc in tincidunt urna. Sed tincidunt eros lacus, sed pulvinar sem venenatis et. Donec euismod orci quis pellentesque sagittis. Donec at tortor in dui mattis facilisis. Pellentesque vel varius lectus. Nunc sed gravida risus, ac finibus elit. Etiam sollicitudin nunc a velit efficitur molestie in ac lectus. Donec vulputate orci odio, sit amet hendrerit odio rhoncus commodo.</stuff>
    <stuff>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ac auctor ex. Nunc in tincidunt urna. Sed tincidunt eros lacus, sed pulvinar sem venenatis et. Donec euismod orci quis pellentesque sagittis. Donec at tortor in dui mattis facilisis. Pellentesque vel varius lectus. Nunc sed gravida risus, ac finibus elit. Etiam sollicitudin nunc a velit efficitur molestie in ac lectus. Donec vulputate orci odio, sit amet hendrerit odio rhoncus commodo.</stuff>
</root>
"""

while True:
    print('Running once...')
    with PySaxonProcessor(license=False) as proc:
        proc.parse_xml(xml_text=xml_str)

    gc.collect()
    sleep(1)

已尝试的解决方法 #

• 使用PyDocumentBuilder替代PySaxonProcessor解析XML，内存泄漏情况无变化。
• 用del关键字删除Saxon处理器和parse_xml()返回值，内存占用仍无改善。

因需XPath 3.0支持，必须使用Saxon而非lxml。

补充测试：循环外实例化处理器仍存在泄漏 #

有人建议提前在循环外实例化PySaxonProcessor可修复泄漏，但测试后仍存在内存增长：

with PySaxonProcessor(license=False) as proc:
    while True:
        print('Running once...')
        proc.parse_xml(xml_text=xml_str)

        gc.collect()
        sleep(1)

提问 #

请问我哪里操作有误？如何用Saxon无泄漏地解析XML？

内容的提问来源于stack exchange，提问作者Rainbolt