Talend ESB：XML校验XSD失败时如何写入拒斥数据至数据库及场景示例

Hey there! Let's break down how to implement this XML validation workflow in Talend ESB step by step—covering everything from setup to testing, so you can get this up and running smoothly.

1. Pre-Requisites First #

Before diving into the workflow, make sure you have these ready:

• A working installation of Talend ESB Studio (v7.x or 8.x is recommended)
• Your target XSD file imported into your Talend project's Resources folder (so it's easily accessible to the route)
• Two database connections configured in Talend's Metadata > DB Connections: one for your central valid message table, another for the failed message table.

2. Build the Core Route (Talend ESB Route) #

We'll use a Talend ESB Route (based on Apache Camel) since it's the most natural fit for ESB-style message routing and validation. Here's the component-by-component setup:

2.1 Inbound Message Receiver #

Start by adding the component that matches your inbound source:

• If receiving XML via HTTP/REST: Use the Restlet component, configure your endpoint (e.g., http://0.0.0.0:8080/xml-submit) and set the HTTP method to POST.
• If receiving XML files: Use the File component, point it to your input directory and set the file filter to *.xml.

2.2 XSD Validation Step #

Add the Validate component right after your inbound receiver:

1. In the component configuration, select Schema Validation as the validation type.
2. Choose XML Schema as the schema language.
3. Browse to your imported XSD file (use classpath:your-schema.xsd if it's in the resources folder).
4. This component will automatically throw a ValidationException if the XML fails to match the XSD.

2.3 Route Messages Based on Validation Result #

We'll use Camel's exception handling and direct routing to split valid and invalid messages:

2.3.1 Handle Valid Messages (Insert to Central DB)

After the Validate component:

1. Add an XPath component (or use Camel's .xpath() DSL) to extract the relevant fields from your XML (e.g., customer ID, order amount).
2. Connect to a JDBC component configured to your central database.
3. Write an INSERT query that maps your extracted fields to the central table columns—don't forget to include the raw XML if you need to retain the full message.

Example snippet for the valid path:

.xpath("//customerId/text()", String.class).setHeader("CUSTOMER_ID")
.xpath("//orderTotal/text()", Double.class).setHeader("ORDER_TOTAL")
.to("jdbc:centralDB?query=INSERT INTO valid_messages (customer_id, order_total, received_at, raw_xml) VALUES (:#CUSTOMER_ID, :#ORDER_TOTAL, NOW(), :#in.body)")
.log("Successfully inserted valid message to central DB")

2.3.2 Handle Invalid Messages (Insert to Failed DB)

Add an exception handler block to catch validation failures:

1. In your route, define an onException(ValidationException.class) block.
2. Inside this block, capture key details: the raw invalid XML, the validation error message, and a timestamp.
3. Use another JDBC component to insert these details into your failed messages table.
4. Add .handled(true) to mark the exception as resolved so it doesn't propagate further.

Example exception handling snippet:

.onException(ValidationException.class)
    .log("XML validation failed: ${exception.message}")
    .setHeader("ERROR_MESSAGE").simple("${exception.message}")
    .setHeader("RECEIVED_AT").simple("${date:now:yyyy-MM-dd HH:mm:ss}")
    .to("jdbc:failedDB?query=INSERT INTO invalid_messages (raw_xml, error_msg, received_at) VALUES (:#in.body, :#ERROR_MESSAGE, :#RECEIVED_AT)")
    .handled(true)
    .log("Logged invalid message to failed DB")

3. Full Route Example (DSL) #

Here's how the complete route might look in Talend's DSL view:

from("restlet:http://0.0.0.0:8080/xml-submit?restletMethod=POST")
    .log("Received inbound XML: ${in.body}")
    .validate().schema("classpath:order-schema.xsd")
    // Valid path
    .xpath("//orderId/text()", String.class).setHeader("ORDER_ID")
    .xpath("//customerName/text()", String.class).setHeader("CUSTOMER_NAME")
    .to("jdbc:centralDB?query=INSERT INTO valid_orders (order_id, customer_name, received_time, raw_xml) VALUES (:#ORDER_ID, :#CUSTOMER_NAME, NOW(), :#in.body)")
    .log("Valid order saved to central DB")
// Exception handling for invalid XML
.onException(ValidationException.class)
    .setHeader("ERROR_DETAILS").simple("${exception.message}")
    .setHeader("TIMESTAMP").simple("${date:now:yyyy-MM-dd HH:mm:ss}")
    .to("jdbc:failedDB?query=INSERT INTO rejected_orders (raw_xml, error_details, timestamp) VALUES (:#in.body, :#ERROR_DETAILS, :#TIMESTAMP)")
    .handled(true)
    .log("Rejected order logged to failed DB");

4. Key Tips for Success #

• XSD Accessibility: Always store your XSD in the project's src/main/resources folder so the route can find it via the classpath: reference.
• Database Connection Testing: Test your DB connections in Talend's Metadata view before building the route to avoid connection issues.
• Raw Message Retention: Saving the full raw XML in both tables makes debugging way easier—you can always reprocess invalid messages later if needed.
• Error Logging: Add detailed logs to each step so you can trace message flow if something goes wrong.

5. Testing the Workflow #

• Valid XML Test: Use Postman or curl to send an XML that matches your XSD. Check your central database table to confirm the entry exists.
• Invalid XML Test: Send an XML with missing required fields or invalid data types. Verify that a row is added to your failed table with the correct error message and raw XML.

内容的提问来源于stack exchange，提问作者MSamuel83