Debezium UI创建连接器遇API错误及自动创建主题告警问题

我使用以下docker-compose配置部署了Kafka、Debezium Connect及Debezium UI：

version: '3'

networks:
  kafka-network:
    driver: bridge

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.6.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    ports:
      - "2181:2181"
    networks:
      - kafka-network

  kafka:
    image: confluentinc/cp-kafka:7.6.0
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "29092:29092"  # Host access
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: true
    networks:
      - kafka-network

  kafka-ui:
    image: provectuslabs/kafka-ui
    depends_on:
      - kafka
    networks:
      - kafka-network
    ports:
      - "8081:8080"
    restart: always
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092

  debezium-connect:
    image: debezium/connect:2.4
    container_name: debezium-connect
    ports:
      - "8083:8083"
    environment:
      GROUP_ID: 1
      CONFIG_STORAGE_TOPIC: debezium_connect_configs
      OFFSET_STORAGE_TOPIC: debezium_connect_offsets
      STATUS_STORAGE_TOPIC: debezium_connect_status
      BOOTSTRAP_SERVERS: kafka:29092
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE: true
      CONNECT_PLUGIN_PATH: "/kafka/connect"
      CONNECT_TOPIC_CREATION_ENABLE: true
    depends_on:
      - kafka
    networks:
      - kafka-network
    healthcheck:
      test:
        [
          'CMD',
          'curl',
          '--silent',
          '--fail',
          '-X',
          'GET',
          'http://localhost:8083/connectors',
        ]
      interval: 10s
      timeout: 5s
      retries: 5

  debezium-ui:
    image: debezium/debezium-ui:latest
    restart: always
    container_name: debezium-ui
    hostname: debezium-ui
    depends_on:
       debezium-connect:
        condition: service_healthy
    networks:
      - kafka-network
    ports:
      - '8085:8080'
    environment:
      KAFKA_CONNECT_URIS: http://debezium-connect:8083
      GROUP_ID: 1
      CONFIG_STORAGE_TOPIC: connect_configs
      OFFSET_STORAGE_TOPIC: connect_offsets
      STATUS_STORAGE_TOPIC: connect_statuses
      ENABLE_DEBEZIUM_KC_REST_EXTENSION: 1
      CONNECT_REST_EXTENSION_CLASSES: io.debezium.kcrestextension.DebeziumConnectRestExtension,io.debezium.connector.mysql.rest.DebeziumMySqlConnectRestExtension,io.debezium.connector.postgresql.rest.DebeziumPostgresConnectRestExtension,io.debezium.connector.mongodb.rest.DebeziumMongoDbConnectRestExtension,io.debezium.connector.oracle.rest.DebeziumOracleConnectRestExtension,io.debezium.connector.sqlserver.rest.DebeziumSqlServerConnectRestExtension
      ENABLE_DEBEZIUM_SCRIPTING: 1
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter

尝试通过http://localhost:8085的Debezium UI创建连接器时，遇到两个问题：

1. 在Transformations步骤出现API错误；
2. 在Topic Creation环节收到告警：Info alert: Kafka automatic topic creation is not enabled. See documentation for details on enabling this feature.

尽管已将CONNECT_TOPIC_CREATION_ENABLE和KAFKA_AUTO_CREATE_TOPICS_ENABLE设置为true，问题仍存在，请问是否需要配置其他变量？

一、解决Topic Creation告警问题 #

1. 移除Debezium UI的冗余配置 #

Debezium UI仅作为前端控制台，不需要配置GROUP_ID、CONFIG_STORAGE_TOPIC等Connect服务专属参数，这些配置会干扰UI的状态检查逻辑。修改debezium-ui的environment部分，保留必要参数即可：

debezium-ui:
  image: debezium/debezium-ui:latest
  restart: always
  container_name: debezium-ui
  hostname: debezium-ui
  depends_on:
     debezium-connect:
      condition: service_healthy
  networks:
    - kafka-network
  ports:
    - '8085:8080'
  environment:
    KAFKA_CONNECT_URIS: http://debezium-connect:8083

2. 补充Connect的主题创建默认参数 #

单节点Kafka集群需要指定默认副本因子和分区数（Kafka默认副本因子为3，单节点无法满足），否则自动创建主题会失败。在debezium-connect的environment中添加：

CONNECT_TOPIC_CREATION_DEFAULT_REPLICATION_FACTOR: 1
CONNECT_TOPIC_CREATION_DEFAULT_PARTITIONS: 1

3. 强化Kafka的自动创建配置 #

在kafka服务的environment中添加额外参数，确保Broker允许自动创建主题：

KAFKA_ALLOW_AUTO_CREATE_TOPICS: true

二、解决Transformations步骤API错误问题 #

1. 迁移REST扩展配置到Connect服务 #

ENABLE_DEBEZIUM_KC_REST_EXTENSION、CONNECT_REST_EXTENSION_CLASSES等是Debezium Connect需要加载的扩展配置，而非UI的配置。将这些参数从debezium-ui移到debezium-connect的environment中：

debezium-connect:
  # ... 其他配置不变
  environment:
    # ... 原有参数不变
    ENABLE_DEBEZIUM_KC_REST_EXTENSION: 1
    CONNECT_REST_EXTENSION_CLASSES: io.debezium.kcrestextension.DebeziumConnectRestExtension,io.debezium.connector.mysql.rest.DebeziumMySqlConnectRestExtension,io.debezium.connector.postgresql.rest.DebeziumPostgresConnectRestExtension,io.debezium.connector.mongodb.rest.DebeziumMongoDbConnectRestExtension,io.debezium.connector.oracle.rest.DebeziumOracleConnectRestExtension,io.debezium.connector.sqlserver.rest.DebeziumSqlServerConnectRestExtension
    ENABLE_DEBEZIUM_SCRIPTING: 1

2. 重启服务生效 #

执行以下命令彻底重启服务（删除卷确保配置完全刷新）：

docker-compose down -v
docker-compose up -d

重启后重新尝试在Debezium UI中创建连接器，两个问题应该都能解决。

内容的提问来源于stack exchange，提问作者Марина Лисниченко