基于指定Avro Schema生成Kafka可用正确JSON样例数据的咨询

错误原因拆解 #

测试阶段Union类型报错 #

• employeeNumber字段为["null", "string"]的Union类型，错误使用{"string": "xxx"}的包裹格式，Avro JSON编码中Union类型的非Null值无需额外包裹类型键，直接传入对应值即可。
• serialNumbers字段为["null", array<string>]的Union类型，错误将数组包装为{"type": "array", "items": [...]}的对象结构，不符合Avro JSON编码规则。

Dataflow作业报错 #

serialNumbers字段被以对象形式传递，而非直接传入数组，导致系统识别为“非重复字段传入数组”，违反Schema定义的Union（null或数组）结构要求。

正确的JSON样例数据 #

{
    "employeeID": "qtete46524",
    "employeeNumber": "custnumber9813",
    "serialNumbers": ["363536623","5846373733"],
    "correlationId": "corr-656532443",
    "timestamp": 1476538955719,
    "employmentscreening": "NO",
    "vouchercodes": [
        {
            "voucherName": "skygo",
            "authocode": "A238472ASD"
        }
    ]
}

含Null值的样例参考 #

若需将Union类型字段设为Null，示例如下：

{
    "employeeID": "qtete46524",
    "employeeNumber": null,
    "serialNumbers": null,
    "correlationId": "corr-656532443",
    "timestamp": 1476538955719,
    "employmentscreening": "YES",
    "vouchercodes": null
}

内容的提问来源于stack exchange，提问作者data2quest