咨询Solr 7.6复杂XML索引的最优方案及替代实现方式

Hey Katja, let's tackle this problem step by step—you've got specific needs (XSLT 2 support + preserving full XML records) that out-of-the-box Solr tools aren't meeting, so here are practical, actionable solutions:

1. Build a Custom DIH EntityProcessor with XSLT 2 Support #

The default XPathEntityProcessor is limited to XSLT 1, but DIH is designed to be extensible. Here's how to add XSLT 2 power:

• Create a custom Java class extending AbstractEntityProcessor (part of Solr's DIH API).
• In the init() or process() method, initialize Saxon's TransformerFactory (instead of the default JAXP one) to handle XSLT 2 stylesheets.
• Process the input XML with your XSLT 2 logic to extract indexed fields, and simultaneously serialize the full raw XML to a string (you can read the input stream or DOM directly) to add as a dedicated field like raw_xml.
• Register your custom processor in solrconfig.xml under your DIH configuration, then reference it in data-config.xml just like any standard entity processor.

This keeps all logic inside Solr, avoids bloated intermediate files, and gives you full XSLT 2 functionality.

2. Use Solr's Scripting Update Processor with XSLT 2 #

If you prefer scripting over Java code, Solr's update processor chain supports custom scripts (like Groovy or JavaScript):

• Drop the Saxon library into Solr's lib directory (make sure it's compatible with your Solr version—Saxon-HE is free and works for most cases).
• Define an update processor in solrconfig.xml that uses a Groovy script.
• In the script, for each incoming XML document:
  1. Capture the raw XML content from the request's stream.
  2. Use Saxon to apply your XSLT 2 transformation and extract fields for indexing.
  3. Add the full raw XML string as a new field (e.g., full_xml_record) to the Solr document.
  4. Pass the modified document along the chain to complete indexing.

This approach is flexible, no compilation needed, and handles both transformation and raw field storage in one step.

3. Preprocess with an External Service (Keep Solr Vanilla) #

If you don't want to modify Solr's internals, set up a lightweight external preprocessing step:

• Write a simple script (Python, Java, Node.js) that reads your XML files. Use an XSLT 2 engine: Saxon-HE for Java, saxon-js for Node, or lxml with Saxon bindings for Python.
• For each XML record, build a Solr document that includes both the transformed fields and the full raw XML (as an escaped string in a field like original_xml).
• Use Solr's Bulk API or the post command-line tool to send these prepped documents to Solr.

This is great for keeping Solr unmodified, and it lets you test and debug your XSLT 2 logic separately from Solr indexing.

4. Extend the UpdateRequestHandler's ContentStreamTransformer #

The default tr parameter uses XSLT 1, but you can create a custom transformer for XSLT 2:

• Implement Solr's ContentStreamTransformer interface, using Saxon to process input XML with your XSLT 2 stylesheet.
• In your implementation, after transforming the XML to Solr's document format, add an extra field containing the raw XML content.
• Register your custom transformer in solrconfig.xml, then use it via the tr parameter in update requests (e.g., curl -X POST -d @input.xml http://solr:8983/solr/core/update?tr=my-xslt2-transformer).

This mimics the out-of-the-box tr workflow but adds the XSLT 2 support and raw field storage you need.

Quick Tips #

• When storing raw XML, escape special characters (like <, >, ") properly to ensure Solr indexes it as a plain string. Use CDATA sections if sending XML to Solr, or escape the string if using JSON.
• Always test with a small dataset first to verify both transformed fields and raw XML are being indexed correctly.

内容的提问来源于stack exchange，提问作者katja