Avro Schema多记录引用类型出现Unknown union branch报错问题

你定义的Avro Schema根节点为数组结构，代表当前Schema是包含com.example.CustomerAddress和com.example.Customer两种类型的联合类型。根据Avro规范，联合类型对应的JSON数据如果要显式指定类型，需要用类型的全限定名作为外层键；如果不指定类型，也可以直接输出对应类型的结构，由Avro自动匹配分支。
你现有测试JSON在外层额外包裹了"Customer": {}结构，Avro会将短名Customer识别为联合类型的分支名，而你定义的联合分支全限定名为com.example.Customer，二者不匹配，因此抛出Unknown union branch Customer异常。

以下三种方案任选其一即可：

• 方案一（改动最小，优先推荐）：修改测试JSON，去掉外层的Customer包裹
  将customer.json内容调整为直接输出Customer对象结构：

  {
    "first_name": "John",
    "middle_name": null,
    "last_name": "Smith",
    "age": 25,
    "height": 177.6,
    "weight": 120.6,
    "automated_email": true,
    "customer_emails": ["ning.chang@td.com", "test@td.com"],
    "customer_address": {
      "address": "21 2nd Street",
      "city": "New York",
      "postcode": "10021",
      "type": "RESIDENTIAL"
    }
  }
  

  修改完成后重新执行原转换命令即可正常运行。

• 方案二：保留现有JSON结构，调整Schema适配JSON格式
  如果你需要保留外层Customer键的结构，可以将Schema调整为根节点是包含Customer字段的包装Record，不再使用联合类型：

  {
    "type": "record",
    "namespace": "com.example",
    "name": "CustomerWrapper",
    "fields": [
      {
        "name": "Customer",
        "type": "com.example.Customer"
      }
    ],
    "types": [
      {
        "type": "record",
        "name": "CustomerAddress",
        "fields": [
          { "name": "address", "type": "string" },
          { "name": "city", "type": "string" },
          { "name": "postcode", "type": ["string", "int"] },
          { "name": "type","type": {"type": "enum","name": "type","symbols": ["POBOX","RESIDENTIAL","ENTERPRISE"]}}
        ]
      },
      {
        "type": "record",
        "name": "Customer",
        "fields": [
          { "name": "first_name", "type": "string" },
          { "name": "middle_name", "type": ["null", "string"], "default": null },       
          { "name": "last_name", "type": "string" },
          { "name": "age", "type": "int" },
          { "name": "height", "type": "float" },
          { "name": "weight", "type": "float" },
          { "name": "automated_email", "type": "boolean", "default": true },
          { "name": "customer_emails", "type": {"type": "array","items": "string"},"default": []},
          { "name": "customer_address", "type": "CustomerAddress" }
        ]
      }
    ]
  }
  

  调整后Schema和现有JSON结构完全匹配，执行原转换命令即可。

• 方案三：保留现有Schema和JSON结构，修改JSON外层键为类型全限定名
  将现有JSON的外层键从Customer改为com.example.Customer，符合Avro联合类型的类型标识规范：

  {
    "com.example.Customer": {
      "first_name": "John",
      "middle_name": null,
      "last_name": "Smith",
      "age": 25,
      "height": 177.6,
      "weight": 120.6,
      "automated_email": true,
      "customer_emails": ["ning.chang@td.com", "test@td.com"],
      "customer_address": {
        "address": "21 2nd Street",
        "city": "New York",
        "postcode": "10021",
        "type": "RESIDENTIAL"
      }
    }
  }
  

  修改完成后执行原转换命令即可正常运行。

内容的提问来源于stack exchange，提问作者Neil