Node.js中如何截取XML指定部分数据,减少xml2js转换量避免崩溃
XML大文件预处理过滤方案
完全可以先过滤无用节点再做JSON转换,大幅降低内存占用避免崩溃,下面给两种可直接落地的实现方案:
方案1:DOM+XPath筛选(适合100M以内XML)
逻辑简单易维护,用XPath直接提取目标节点生成新的小体积XML再转换:
首先安装依赖:
npm install xmldom xpath
示例代码:
const { DOMParser, XMLSerializer } = require('xmldom'); const xpath = require('xpath'); // 替换为你从API拿到的原始XML字符串 const rawXml = '这里填原始XML内容'; const originDoc = new DOMParser().parseFromString(rawXml); const filteredDoc = new DOMParser().parseFromString('<odds></odds>'); const filteredRoot = filteredDoc.documentElement; // 遍历所有event节点 const allEvents = xpath.select('//event', originDoc); allEvents.forEach(originEvent => { // 提取当前event下第一个slug为winner的market节点 const targetMarket = xpath.select1('./market[@slug="winner"][1]', originEvent); if (targetMarket) { // 生成新的event节点,仅保留目标market const newEvent = originEvent.cloneNode(false); newEvent.appendChild(targetMarket.cloneNode(true)); filteredRoot.appendChild(newEvent); } }); // 得到过滤后的XML字符串,再传给xml2js转换即可 const filteredXml = new XMLSerializer().serializeToString(filteredDoc);
方案2:流解析筛选(适合100M以上超大XML)
边读边处理,全程内存占用只有几十KB,完全不会出现内存溢出问题:
首先安装依赖:
npm install sax
示例代码:
const sax = require('sax'); const fs = require('fs'); // 原始XML可以是本地文件也可以是API返回的流 const rawXmlStream = fs.createReadStream('你的原始XML路径'); const writeStream = fs.createWriteStream('./filtered.xml'); const parser = sax.createStream(true); let inEvent = false; let hasFoundTargetMarket = false; // 写入过滤后XML的根节点开头 writeStream.write('<odds>'); parser.on('opentag', node => { if (node.name === 'event') { inEvent = true; hasFoundTargetMarket = false; // 写入event开始标签 const attrs = Object.entries(node.attributes).map(([k, v]) => `${k}="${v}"`).join(' '); writeStream.write(`<event ${attrs}>`); } else if (inEvent && node.name === 'market' && !hasFoundTargetMarket && node.attributes.slug === 'winner') { hasFoundTargetMarket = true; // 写入目标market开始标签 const attrs = Object.entries(node.attributes).map(([k, v]) => `${k}="${v}"`).join(' '); writeStream.write(`<market ${attrs}>`); } else if (hasFoundTargetMarket) { // 写入目标market下的所有子节点 const attrs = Object.entries(node.attributes).map(([k, v]) => `${k}="${v}"`).join(' '); writeStream.write(`<${node.name} ${attrs}>`); } }); parser.on('closetag', tagName => { if (tagName === 'event') { inEvent = false; writeStream.write('</event>'); } else if (tagName === 'market' && hasFoundTargetMarket) { writeStream.write('</market>'); } else if (hasFoundTargetMarket) { writeStream.write(`</${tagName}>`); } }); parser.on('end', () => { writeStream.write('</odds>'); writeStream.end(); // 此时生成的filtered.xml就是仅保留需要节点的小体积XML,直接传给xml2js转换即可 }); rawXmlStream.pipe(parser);
两种方案都可以把需要处理的XML体积降低到原来的几分之一甚至几十分之一,完全可以避免xml2js转换时的崩溃问题。
内容的提问来源于stack exchange,提问作者willmahon
相关产品推荐
相关产品推荐

