使用kafka-schema-registry-maven-plugin处理嵌套Avro Schema报错求助

问题背景 #

项目包含三个Avro Schema文件：MetaModel.avsc、RevisionModel.avsc、RecentChangeEventModel.avsc，其中RecentChangeEventModel.avsc以嵌套对象形式引用另外两个Schema，示例代码如下：

{
  "namespace": "org.taulin.model",
  "type": "record",
  "name": "RecentChangeEvent",
  "fields": [
    {
      "name": "schema",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "meta",
      "type": [
        "null",
        "Meta"
      ],
      "default": null
    },
    {
      "name": "id",
      "type": [
        "null",
        "long"
      ],
      "default": null
    },
    {
      "name": "type",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "namespace",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "title",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "titleUrl",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "comment",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "timestamp",
      "type": [
        "null",
        "long"
      ],
      "default": null
    },
    {
      "name": "user",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "bot",
      "type": [
        "null",
        "boolean"
      ],
      "default": null
    },
    {
      "name": "notifyUrl",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "serverUrl",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "serverName",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "serverScriptPath",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "wiki",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "parsedComment",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "minor",
      "type": [
        "null",
        "boolean"
      ],
      "default": null
    },
    {
      "name": "patrolled",
      "type": [
        "null",
        "boolean"
      ],
      "default": null
    },
    {
      "name": "length",
      "type": [
        "null",
        "Revision"
      ],
      "default": null
    },
    {
      "name": "revision",
      "type": [
        "null",
        "Revision"
      ],
      "default": null
    }
  ]
}

使用kafka-schema-registry-maven-plugin（7.7.1版本）配置Schema注册，配置如下：

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.7.1</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://localhost:8081/</param>
        </schemaRegistryUrls>
        <subjects>
            <Meta>${project.basedir}/src/main/avro/MetaModel.avsc</Meta>
            <Revision>${project.basedir}/src/main/avro/RevisionModel.avsc</Revision>
            <RecentChangeEvent>${project.basedir}/src/main/avro/RecentChangeEventModel.avsc</RecentChangeEvent>
        </subjects>
        <schemaTypes>
            <Meta>AVRO</Meta>
            <Revision>AVRO</Revision>
            <RecentChangeEvent>AVRO</RecentChangeEvent>
        </schemaTypes>
        <references>
            <order>
                <reference>
                    <name>Meta</name>
                    <subject>Meta</subject>
                </reference>
                <reference>
                    <name>Revision</name>
                    <subject>Revision</subject>
                </reference>
                <reference>
                    <name>RecentChangeEvent</name>
                    <subject>RecentChangeEvent</subject>
                </reference>
            </order>
        </references>
    </configuration>
    <goals>
        <goal>register</goal>
    </goals>
</plugin>

执行mvn schema-registry:register时触发错误：

[ERROR] Could not parse Avro schema
org.apache.avro.SchemaParseException: Undefined name: "org.taulin.model.Meta"
    at org.apache.avro.Schema.parse (Schema.java:1697)
    at org.apache.avro.Schema.parse (Schema.java:1823)
    at org.apache.avro.Schema.parse (Schema.java:1736)
    at org.apache.avro.Schema$Parser.parse (Schema.java:1471)

错误原因 #

插件解析RecentChangeEvent Schema时，无法自动找到依赖的Meta和Revision Schema。原有配置中的references.order配置方式不正确，未针对依赖关系做精准关联，且插件默认不会自动扫描加载其他Schema文件。

解决方案 #

方案1：调整插件配置，精准配置Schema引用 #

修改插件配置，确保依赖的Schema先注册，并为RecentChangeEvent配置明确的引用关系：

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.7.1</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://localhost:8081/</param>
        </schemaRegistryUrls>
        <!-- 按依赖顺序注册Schema：先Meta、Revision，再RecentChangeEvent -->
        <subjects>
            <Meta>${project.basedir}/src/main/avro/MetaModel.avsc</Meta>
            <Revision>${project.basedir}/src/main/avro/RevisionModel.avsc</Revision>
            <RecentChangeEvent>${project.basedir}/src/main/avro/RecentChangeEventModel.avsc</RecentChangeEvent>
        </subjects>
        <schemaTypes>
            <Meta>AVRO</Meta>
            <Revision>AVRO</Revision>
            <RecentChangeEvent>AVRO</RecentChangeEvent>
        </schemaTypes>
        <!-- 为RecentChangeEvent配置指向已注册依赖的引用 -->
        <subjectReferences>
            <RecentChangeEvent>
                <reference>
                    <name>org.taulin.model.Meta</name>
                    <subject>Meta</subject>
                    <version>LATEST</version>
                </reference>
                <reference>
                    <name>org.taulin.model.Revision</name>
                    <subject>Revision</subject>
                    <version>LATEST</version>
                </reference>
            </RecentChangeEvent>
        </subjectReferences>
    </configuration>
    <goals>
        <goal>register</goal>
    </goals>
</plugin>

• subjectReferences用于为特定subject配置依赖引用，指定依赖Schema的全限定名、对应的subject名称和版本
• 注册顺序确保依赖Schema先被注册到Registry，避免引用时找不到

方案2：在Schema文件中添加import语句 #

在RecentChangeEventModel.avsc中添加imports字段，显式引用依赖的Schema文件，同时配置插件的Schema目录：

1. 修改RecentChangeEventModel.avsc：

{
  "namespace": "org.taulin.model",
  "type": "record",
  "name": "RecentChangeEvent",
  "imports": [
    "MetaModel.avsc",
    "RevisionModel.avsc"
  ],
  "fields": [
    // 原有字段内容
  ]
}

2. 修改插件配置，添加Schema目录：

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.7.1</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://localhost:8081/</param>
        </schemaRegistryUrls>
        <!-- 指定Schema文件所在目录，插件会自动加载依赖的import文件 -->
        <schemaDirectory>${project.basedir}/src/main/avro</schemaDirectory>
        <subjects>
            <RecentChangeEvent>${project.basedir}/src/main/avro/RecentChangeEventModel.avsc</RecentChangeEvent>
        </subjects>
        <schemaTypes>
            <RecentChangeEvent>AVRO</RecentChangeEvent>
        </schemaTypes>
    </configuration>
    <goals>
        <goal>register</goal>
    </goals>
</plugin>

• imports字段让Avro解析器自动加载指定的依赖Schema文件
• schemaDirectory告诉插件从指定目录查找依赖的Schema文件，无需单独注册依赖Schema（插件会自动处理引用关系并注册所有相关Schema）

方案3：合并所有Schema到单个文件（临时应急方案） #

将Meta、Revision和RecentChangeEvent的Schema定义合并到同一个.avsc文件中，避免跨文件引用问题。此方案适合快速验证，但不利于Schema的维护和版本管理，不推荐长期使用。

内容的提问来源于stack exchange，提问作者Paulo Alves