Kafka Connect File Pulse保留XML属性配置求助

Hey there! I totally get how frustrating it is when critical XML attributes vanish during processing—especially when you're still getting up to speed with Kafka Connect and File Pulse. Let's get those unit attributes back in your payload.

The core issue here is that File Pulse's default XML reader only extracts the text content of elements by default, ignoring attributes. But luckily, there's a simple configuration toggle to enable attribute retention. Here's how to set it up:

Key Configuration to Enable XML Attributes #

Add these lines to your File Pulse Connector configuration to tell the XML parser to preserve attributes:

# Enable extraction of XML attributes
xml.parser.attributes.enabled=true

# Optional: Add a prefix to attribute keys to avoid conflicts with element text
# (e.g., turns `unit="string1"` into `@unit: "string1"`)
xml.parser.attributes.prefix=@

Full Connector Configuration Example #

Here's a complete snippet tailored to your use case:

name=xml-value-attribute-connector
connector.class=io.streamthoughts.kafka.connect.filepulse.source.FilePulseSourceConnector
tasks.max=1
fs.scan.directory.path=/your/xml/file/directory
fs.scan.interval.ms=30000
fs.scan.filters=io.streamthoughts.kafka.connect.filepulse.scanner.local.filter.RegexFileListFilter
file.filter.regex.pattern=.*\.xml$
fs.cleanup.policy.class=io.streamthoughts.kafka.connect.filepulse.clean.LogCleanupPolicy

# Explicitly use the XML file reader
fs.reader.class=io.streamthoughts.kafka.connect.filepulse.reader.XMLFileInputReader

# Enable attribute extraction
xml.parser.attributes.enabled=true
xml.parser.attributes.prefix=@

What This Does #

• xml.parser.attributes.enabled=true: Tells the XML parser to include all element attributes in the output payload.
• xml.parser.attributes.prefix=@: Adds a prefix to attribute keys (like @unit) to distinguish them from the element's text content. For example, a tag like <value unit="string1">sample text</value> will parse to:

  {
    "value": "sample text",
    "@unit": "string1"
  }
  

Troubleshooting Tips #

• Double-check that you're using the XMLFileInputReader (via the fs.reader.class setting)—this configuration only applies to the XML-specific reader.
• If you need to target specific attributes instead of all, combine this with an XPath expression using xml.parser.xpath.expression. For example:

  xml.parser.xpath.expression=//value/* | //value/@unit
  

  This would extract both child elements of <value> and the unit attribute.

Hope this gets those string1/string2 values showing up in your payloads! Let me know if you hit any snags.

内容的提问来源于stack exchange，提问作者soMuchToLearnAndShare