You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Docker部署Druid连接Kerberos认证Kafka失败求助

问题:Druid连接Kerberos认证Kafka失败,无法获取认证信息

机器1部署了带Kerberos认证的Kafka和ZooKeeper,机器2通过Docker Compose部署Druid,已创建自定义镜像与服务编排文件,但连接Kafka Topic时出现以下错误,尝试所有可能配置、查阅相关资料后仍未解决,请求排查配置问题。

错误信息:

Unable to create RecordSupplier: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user

排查方向与问题点

  • Kerberos配置域名不统一:krb5.conf中default_realm为EXAMPLE.COM,但realms段内的KDC地址使用kdc1.domain.com,domain_realm映射example.com=domain.com,存在域名与realm不匹配的问题。Kerberos对realm大小写和域名严格匹配,需将KDC地址改为kdc1.example.com,domain_realm设置为example.com=EXAMPLE.COM。

  • Principal与Keytab不匹配:JAAS配置中的principal为principal@domain.com,但Dockerfile中kinit使用的是user/service@domain.COM,realm大小写不一致(domain.com vs domain.COM),需统一两者的realm格式,确保principal与keytab中存储的主体完全一致。

  • JAAS配置加载不完整:

    • Dockerfile中设置了JAVA_OPTS,但Docker Compose中部分服务又通过DRUID_OPTS传递JVM参数,可能存在参数覆盖或未生效的情况,建议所有Druid服务统一使用DRUID_OPTS指定-Djava.security.auth.login.config=/etc/security/jaas.conf,避免参数冲突。
    • 运行Kafka索引任务的MiddleManager服务是核心,需确保其容器内的keytab文件权限为可读(可在容器内执行ls -l /etc/security/keytab.keytab检查),同时确认JAAS配置文件路径正确。
  • Docker镜像中Kerberos配置覆盖问题:Dockerfile先从builder阶段复制/etc/krb5.conf,随后又用stag/krb5.conf覆盖,需确认stag目录下的krb5.conf是正确的配置文件,避免覆盖后引入错误配置。

  • Kafka网络与服务名匹配:确认Kafka broker的listeners配置中SASL服务名与Druid配置的druid.kafka.consumer.sasl.kerberos.service.name=kafka一致;同时检查机器2的Docker容器是否能访问机器1的Kafka端口(网络连通性、防火墙设置)。

相关配置文件

Dockerfile

FROM debian:bookworm-slim AS builder

RUN apt-get update && apt-get install -y \
    krb5-user \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

FROM apache/druid:30.0.0

COPY --from=builder /usr/bin/kinit /usr/bin/kinit
COPY --from=builder /usr/bin/klist /usr/bin/klist
COPY --from=builder /etc/krb5.conf /etc/krb5.conf
COPY --from=builder /usr/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/

COPY stag/krb5.conf /etc/krb5.conf
COPY stag/jaas.conf /etc/security/jaas.conf
COPY stag/keytab.keytab /etc/security/keytab.keytab

ENV JAVA_OPTS=\"-Djava.security.auth.login.config=/etc/security/jaas.conf\"

CMD [\"/bin/bash\", \"-c\", \"kinit -kt /etc/security/keytab.keytab user/service@domain.COM && druid/bin/start-micro-quickstart\"]

Docker Compose

version: "2.2"

volumes:
  metadata_data: {}
  middle_var: {}
  historical_var: {}
  broker_var: {}
  coordinator_var: {}
  router_var: {}
  druid_shared: {}
  overlord_var: {}

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  postgres:
    container_name: postgres
    image: postgres:latest
    ports:
      - "5432:5432"
    volumes:
      - metadata_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=SecretPassword
      - POSTGRES_USER=user
      - POSTGRES_DB=exampledb

  coordinator:
    image: my-druid-with-kerberos:latest
    container_name: coordinator
    volumes:
      - druid_shared:/opt/shared
      - coordinator_var:/opt/druid/var
    depends_on:
      - postgres
    ports:
      - "8081:8081"
    command:
      - coordinator
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

  broker:
    image: my-druid-with-kerberos:latest
    container_name: broker
    volumes:
      - broker_var:/opt/druid/var
      - /path/to/krb5.conf:/etc/krb5.conf
      - /path/to/jaas.conf:/etc/security/jaas.conf
      - /path/to/keytab.keytab:/etc/security/keytab.keytab
    depends_on:
      - postgres
      - coordinator
    ports:
      - "8082:8082"
    command:
      - broker
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

  historical:
    image: my-druid-with-kerberos:latest
    container_name: historical
    volumes:
      - druid_shared:/opt/shared
      - historical_var:/opt/druid/var
      - /path/to/krb5.conf:/etc/krb5.conf
      - /path/to/jaas.conf:/etc/security/jaas.conf
      - /path/to/keytab.keytab:/etc/security/keytab.keytab
    depends_on:
      - postgres
      - coordinator
    ports:
      - "8083:8083"
    command:
      - historical
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

  middlemanager:
    image: my-druid-with-kerberos:latest
    container_name: middlemanager
    volumes:
      - druid_shared:/opt/shared
      - middle_var:/opt/druid/var
      - /path/to/krb5.conf:/etc/krb5.conf
      - /path/to/jaas.conf:/etc/security/jaas.conf
      - /path/to/keytab.keytab:/etc/security/keytab.keytab
    depends_on:
      - postgres
      - coordinator
    ports:
      - "8091:8091"
      - "8100-8105:8100-8105"
    command:
      - middleManager
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

  router:
    image: my-druid-with-kerberos:latest
    container_name: router
    volumes:
      - router_var:/opt/druid/var
      - /path/to/krb5.conf:/etc/krb5.conf
      - /path/to/jaas.conf:/etc/security/jaas.conf
      - /path/to/keytab.keytab:/etc/security/keytab.keytab
    depends_on:
      - postgres
      - coordinator
    ports:
      - "8888:8888"
    command:
      - router
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

  overlord:
    image: my-druid-with-kerberos:latest
    container_name: overlord
    volumes:
      - druid_shared:/opt/shared
      - overlord_var:/opt/druid/var
      - /path/to/krb5.conf:/etc/krb5.conf
      - /path/to/jaas.conf:/etc/security/jaas.conf
      - /path/to/keytab.keytab:/etc/security/keytab.keytab
    depends_on:
      - postgres
    ports:
      - "8090:8090"
    command:
      - overlord
    env_file:
      - environment
    environment:
      - druid.zk.service.host=zookeeper:2181
      - DRUID_OPTS=-Djava.security.auth.login.config=/etc/security/jaas.conf

Druid环境变量文件

DRUID_SINGLE_NODE_CONF=micro-quickstart

druid_emitter_logging_logLevel=debug

druid_extensions_loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "postgresql-metadata-storage", "druid-multi-stage-query","druid-kafka-indexing-service"]

druid_zk_service_host=zookeeper

druid_metadata_storage_host=
druid_metadata_storage_type=postgresql
druid_metadata_storage_connector_connectURI=jdbc:postgresql://postgres:5432/exampledb
druid_metadata_storage_connector_user=user
druid_metadata_storage_connector_password=SecretPassword

druid_coordinator_balancer_strategy=cachingCost

druid_indexer_runner_javaOptsArray=["-server", "-Xmx1g", "-Xms1g", "-XX:MaxDirectMemorySize=3g", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8", "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager"]
druid_indexer_fork_property_druid_processing_buffer_sizeBytes=256MiB

druid_storage_type=local
druid_storage_storageDirectory=/opt/shared/segments
druid_indexer_logs_type=file
druid_indexer_logs_directory=/opt/shared/indexing-logs

druid_processing_numThreads=2
druid_processing_numMergeBuffers=2

druid.kafka.consumer.security.protocol=SASL_PLAINTEXT
druid.kafka.consumer.sasl.mechanism=GSSAPI
druid.kafka.consumer.sasl.kerberos.service.name=kafka

JAAS配置文件

KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
    useTicketCache=false
    useKeyTab=true
    storeKey=true
    debug=true
    serviceName="kafka"
    keyTab="/etc/security/keytab.keytab"
    principal="principal@domain.com";
};

Kerberos配置文件

[libdefaults]
    ccache_type = 4
    default_realm = EXAMPLE.COM
    fcc-mit-ticketflags = true
    forwardable = true
    kdc_timesync = 1
    proxiable = true
    renew_lifetime = 30d
    ticket_lifetime = 3d

[realms]
    EXAMPLE.COM = {
        admin_server = kdc1.example.com
        default_domain = example.com
        kdc = kdc1.domain.com
        kdc = kdc2.domain.com
        kdc = kdc3.domain.com
    }

[domain_realm]
    example.com = domain.com

内容的提问来源于stack exchange,提问作者hektor102

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.06.19 01:43:10