如何通过Curl命令成功向Confluent Schema Registry注册JSON Schema？

问题 #

尝试使用curl向Confluent Schema Registry注册JSON Schema时持续报错，错误码42201，提示schema无效，但使用kafka-json-schema-console-producer工具却能成功注册。

报错的curl命令 #

echo '{
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "age": {
      "type": "integer"
    },
    "profession": {
      "type": "string"
    },
    "Status": {
      "type": "string"
    },
    "education": {
      "type": "string"
    }
  }
}' | \
jq '. | {schema: tojson}' | \
curl -X POST "https://schema-registry.dev/subjects/cld-dev-pub-json-test-value/versions" -H "Content-Type:application/json" -d @-

报错信息 #

{
  "error_code": 42201,
  "message": "Either the input schema or one its references is invalid"
}

成功的kafka-json-schema-console-producer命令 #

kafka-json-schema-console-producer --bootstrap-server kafka00.dev:9095 --producer.config kafka-client-dev.properties --topic cld-dev-pub-json-test --property schema.registry.url=https://schema-registry.dev --property basic.auth.credentials.source=USER_INFO --property schema.registry.basic.auth.user.info=username:password --property value.schema='{"type":"object","properties":{"id":{"type":"integer"} , "age":{"type":"integer"} , "profession":{"type":"string"},"Status" : {"type": "string" }, "education" : {"type": "string" }}}' < json_data.txt

查询Registry中的Schema结果 #

curl -s -u <USERNAME:PASSWORD> https://schema-registry.dev.kafka/subjects/cld-dev-pub-json-test-value/versions/latest | jq

返回结果：

{
  "subject": "cld-dev-pub-json-test-value",
  "version": 8,
  "id": 2639,
  "schemaType": "JSON",
  "schema": "{\"type\":\"object\",\"properties\":{\"id\":{\"type\":\"integer\"},\"age\":{\"type\":\"integer\"},\"profession\":{\"type\":\"string\"},\"Status\":{\"type\":\"string\"},\"education\":{\"type\":\"string\"}}}"
}

解决方案 #

问题根源有两个：curl命令缺少认证信息，以及JSON Schema的转义处理不正确。

修正后的curl命令 #

echo '{
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "age": {
      "type": "integer"
    },
    "profession": {
      "type": "string"
    },
    "Status": {
      "type": "string"
    },
    "education": {
      "type": "string"
    }
  }
}' | \
jq -c '{schema: tostring}' | \
curl -X POST "https://schema-registry.dev/subjects/cld-dev-pub-json-test-value/versions" \
  -H "Content-Type:application/json" \
  -u <USERNAME:PASSWORD> \
  -d @-

关键修正点说明 #

• 添加认证：加上-u <USERNAME:PASSWORD>参数，和查询命令保持一致。Schema Registry开启了基础认证，未携带认证信息会导致服务端返回的错误信息不准确。
• 调整jq处理逻辑：将tojson改为tostring，确保schema字段的值是仅转义内部引号的原始JSON Schema字符串。tojson会把整个Schema对象再次转成JSON字符串，导致服务端接收到双重转义的无效内容；tostring则直接将原始JSON结构转为符合要求的字符串格式。
• 修复JSON格式：将原echo命令中的HTML实体"替换为正常的双引号，确保输入的JSON本身合法。

内容的提问来源于stack exchange，提问作者user6702954