Java XML序列化未过滤无效字符致反序列化失败的解决方法

我在Java 17中序列化XML文档到文件，后续反序列化时抛出SAXParseException，原因是文档包含XML无效字符。原本预期Java序列化的文档能被自身反序列化，现在有两个核心问题：

1. 如何读取已生成的包含无效字符的XML文件？
2. 如何避免新生成的XML文件包含无效字符？

最小复现示例 #

package example;

import static java.nio.charset.StandardCharsets.US_ASCII;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

import org.w3c.dom.Document;
import org.w3c.dom.Element;

public class Scratch {
    public static void main(String[] args) throws Exception {
        // \u001b = 转义字符，XML 1.0中不允许
        Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument();
        Element element = document.createElement("tag");
        element.setAttribute("attribute", "\u001b");
        document.appendChild(element);

        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        TransformerFactory transformerFactory = TransformerFactory.newInstance();
        System.out.println("transformerFactory.getClass() = " + transformerFactory.getClass());
        // transformerFactory.getClass() = class org.apache.xalan.processor.TransformerFactoryImpl
        Transformer transformer = transformerFactory.newTransformer();
        transformer.setOutputProperty(OutputKeys.ENCODING, US_ASCII.name());
        transformer.transform(new DOMSource(document), new StreamResult(bos));
        String xmlString = bos.toString(US_ASCII);

        System.out.println("xmlString = " + xmlString);
        // xmlString = <?xml version="1.0" encoding="US-ASCII"?><tag attribute="&#27;"/>

        DocumentBuilder domDocumentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
        domDocumentBuilder.parse(new ByteArrayInputStream(xmlString.getBytes(US_ASCII)));
        // [Fatal Error] :1:63: Character reference "&#27" is an invalid XML character.
        // Exception in thread "main" org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 63; Character reference "&#27" is an invalid XML character.
        // at java.xml/com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:262)
        // at java.xml/com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:342)
        // at java.xml/javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:122)
        // at example.Scratch.main(Scratch.java:40)
    }
}

问题根源 #

默认使用的Xalan转换器在序列化时，会把XML无效字符转成字符引用（比如），但这类字符本身不符合XML 1.0规范，导致解析器报错。

一、读取已生成的含无效字符的XML文件 #

要解析这类文件，需要预处理XML内容，替换掉无效的字符引用：

import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

// 读取文件内容
String xmlContent = Files.readString(Paths.get("invalid.xml"), US_ASCII);
// 匹配并替换XML 1.0无效的字符引用
Pattern invalidCharRefs = Pattern.compile("&#(?:0?[0-8BCEF]|1[0-9A-F]|7F|9[0-F]);", Pattern.CASE_INSENSITIVE);
Matcher matcher = invalidCharRefs.matcher(xmlContent);
String cleanedXml = matcher.replaceAll("�");

// 解析清理后的内容
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(new ByteArrayInputStream(cleanedXml.getBytes(US_ASCII)));

二、防止新生成的XML包含无效字符 #

不想遍历整个DOM树的话，可选择以下两种高效方案：

方案1：使用支持自动过滤的转换器 #

改用Saxon转换器（需引入依赖），它会自动处理无效字符，无需手动遍历DOM：

Maven依赖：

<dependency>
    <groupId>net.sf.saxon</groupId>
    <artifactId>Saxon-HE</artifactId>
    <version>12.4</version>
</dependency>

使用代码：

TransformerFactory transformerFactory = new net.sf.saxon.TransformerFactoryImpl();
Transformer transformer = transformerFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, US_ASCII.name());
transformer.transform(new DOMSource(document), new StreamResult(bos));

Saxon会自动将无效字符替换为合法的替代字符，生成可正常解析的XML。

方案2：序列化时实时过滤无效字符 #

通过自定义FilterOutputStream包装输出流，在序列化阶段实时过滤无效字符：

ByteArrayOutputStream bos = new ByteArrayOutputStream();
FilterOutputStream filteredOut = new FilterOutputStream(bos) {
    private final Pattern invalidChars = Pattern.compile("[\\x00-\\x08\\x0B\\x0C\\x0E-\\x1F\\x7F-\\x9F\\uFEFF\\uFFFE\\uFFFF]");

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
        String content = new String(b, off, len, US_ASCII);
        Matcher matcher = invalidChars.matcher(content);
        String cleaned = matcher.replaceAll("�");
        super.write(cleaned.getBytes(US_ASCII));
    }
};

Transformer transformer = transformerFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, US_ASCII.name());
transformer.transform(new DOMSource(document), new StreamResult(filteredOut));

兜底方案：遍历DOM清理无效字符（已验证可行） #

如果上述方案不适用，遍历DOM树清理所有节点和属性中的无效字符也是可靠方式：

package example;

import static java.nio.charset.StandardCharsets.US_ASCII;
import static org.w3c.dom.Node.CDATA_SECTION_NODE;
import static org.w3c.dom.Node.COMMENT_NODE;
import static org.w3c.dom.Node.ELEMENT_NODE;
import static org.w3c.dom.Node.TEXT_NODE;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;

class Scratch {

    private static final Pattern VALID_CHARACTERS = Pattern.compile(
            "[\\u0009\\u000a\\u000d\\u0020-\\uD7FF\\uE000-\\uFFFD]+");
    private static final Pattern INVALID_XML_CHARACTERS = Pattern.compile(
            "(?<![\\uD800-\\uDBFF])[\\uDC00-\\uDFFF]|[\\uD800-\\uDBFF](?![\\uDC00-\\uDFFF])|[\\x00-\\x08\\x0B\\x0C\\x0E-\\x1F\\x7F-\\x9F\\uFEFF\\uFFFE\\uFFFF]");
    private static final String REPLACEMENT_CHARACTER = "�";

    public static void main(String[] args) throws Exception {
        // \u001b = 转义字符，XML 1.0中不允许
        Document document = DocumentBuilderFactory.newInstance().newDocumentBuilder().newDocument();
        Element element = document.createElement("tag");
        element.setAttribute("attribute", "\u001b");
        document.appendChild(element);

        stripInvalidXmlCharacters(document);

        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        TransformerFactory transformerFactory = TransformerFactory.newInstance();
        System.out.println("transformerFactory.getClass() = " + transformerFactory.getClass());
        final Transformer transformer = transformerFactory.newTransformer();
        transformer.setOutputProperty(OutputKeys.ENCODING, US_ASCII.name());
        transformer.transform(new DOMSource(document), new StreamResult(bos));
        String xmlString = bos.toString(US_ASCII);

        System.out.println("xmlString = " + xmlString);

        DocumentBuilder domDocumentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();
        domDocumentBuilder.parse(new ByteArrayInputStream(xmlString.getBytes(US_ASCII)));
    }

    private static void stripInvalidXmlCharacters(Node node) {
        NamedNodeMap attributes = node.getAttributes();
        if (attributes != null) {
            int length = attributes.getLength();
            for (int i = 0; i < length; i++) {
                Node attribute = attributes.item(i);
                attribute.setNodeValue(stripInvalidXmlCharacters(attribute.getNodeValue()));
            }
        }
        for (Node child = node.getFirstChild(); child != null; child = child.getNextSibling()) {
            switch (child.getNodeType()) {
                case ELEMENT_NODE -> stripInvalidXmlCharacters(child);
                case TEXT_NODE, CDATA_SECTION_NODE, COMMENT_NODE ->
                        child.setNodeValue(stripInvalidXmlCharacters(child.getNodeValue()));
            }
        }
    }

    private static String stripInvalidXmlCharacters(String input) {
        if (input == null || input.isEmpty() || VALID_CHARACTERS.matcher(input).matches()) {
            return input;
        }
        Matcher m = INVALID_XML_CHARACTERS.matcher(input);
        StringBuilder sb = new StringBuilder();
        while (m.find()) {
            m.appendReplacement(sb, REPLACEMENT_CHARACTER);
        }
        m.appendTail(sb);
        return sb.toString();
    }
}

内容的提问来源于stack exchange，提问作者Kristof Neirynck