GitLab流水线中Karate容器无法访问Kafka-Rest的问题排查

我在GitLab流水线中使用docker/compose:latest镜像运行以下脚本构建并启动容器：

script:
    - docker-compose up -d
    - CONTAINER_ID=$(docker-compose ps -q karate)
    - EXIT_CODE=$(docker inspect -f '{{.State.ExitCode}}' $CONTAINER_ID)
    - |
      if [ $EXIT_CODE -eq 0 ]; then
        echo "Tests passed!"
      else
        echo "Tests failed with exit code: $EXIT_CODE"
        exit 1
      fi

本地执行docker-compose up -d时，Karate测试可以正常访问API（http://api:35000/）和Kafka-Rest（http://kafkarest:38082）；但在流水线中，测试只能访问API，请求http://kafkarest:38082/consumers/testgroup时出现连接拒绝错误：

Connect to kafkarest:38082 [kafkarest/172.20.0.12] failed: Connection refused (Connection refused), http call failed after 43 milliseconds for url: http://kafkarest:38082/consumers/testgroup

试过http://kafkarest/、http://docker:38082、http://localhost等地址都无效，请问应该用哪个URL连接Kafka-Rest？

我的docker-compose配置如下：

api:
    container_name: 'api'
    depends_on:
      - "db"
      - "kafka"
    build:
      dockerfile: Dockerfile
      context: .
    ports:
      - 35000:80
    environment:
      - ASPNETCORE_ENVIRONMENT=Docker
    networks:
      - capy-network

  karate:
    build:
      context: .
      dockerfile: Dockerfile-karate
    environment:
      - KARATE_ENV=mr
    depends_on:
      - db
      - s3
      - api
      - kafkarest 
    networks:
      - capy-network

  zookeeper:
    image: confluentinc/cp-zookeeper:4.1.1
    container_name: 'zookeeper'
    hostname: zookeeper
    ports: 
    - "32181:32181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_SYNC_LIMIT: 2
    networks:
      - capy-network

  kafka:
    image: confluentinc/cp-kafka:4.1.1
    container_name: 'kafka'
    hostname: kafka
    ports: 
    - "29092:29092"
    depends_on:
    - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    networks:
      - capy-network

  kafka-ui:
    image: provectuslabs/kafka-ui
    container_name: 'kafka-ui'
    ports:
      - 8090:8080
    restart: always
    environment:
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:29092
      - KAFKA_CLUSTERS_0_ZOOKEEPER=zookeeper:32181
    links:
      - kafka
      - zookeeper 
    networks:
      - capy-network  

  schema-registry:
    image: confluentinc/cp-schema-registry:4.1.1
    container_name: 'schema-registry'
    hostname: schema-registry
    ports:
    - "38081:38081"
    depends_on:
    - kafka
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:32181
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_LISTENERS: http://schema-registry:38081
      SCHEMA_REGISTRY_DEBUG: "true"
    networks:
      - capy-network

  kafkarest:
    image: confluentinc/cp-kafka-rest:4.1.1
    container_name: 'kafkarest'
    hostname: kafkarest
    ports:
    - "38082:38082"
    depends_on:
      - schema-registry
    environment:
      KAFKA_REST_ZOOKEEPER_CONNECT: zookeeper:32181
      KAFKA_REST_SCHEMA_REGISTRY_URL: schema-registry:38081
      KAFKA_REST_HOST_NAME: kafkarest
      KAFKA_REST_LISTENERS: http://kafkarest:38082
    networks:
      - capy-network

核心原因 #

depends_on仅保证容器启动顺序，不确保内部服务完全就绪。流水线环境中，kafkarest容器虽已启动，但服务初始化未完成，导致Karate测试请求时连接被拒绝。

解决步骤 #

1. 添加服务就绪等待逻辑
   修改流水线脚本，在启动容器后轮询kafkarest服务，直到其就绪再执行测试：

   script:
       - docker-compose up -d
       # 轮询等待kafkarest服务可用
       - until curl -s http://kafkarest:38082/topics; do sleep 5; echo "Waiting for kafkarest..."; done
       - CONTAINER_ID=$(docker-compose ps -q karate)
       - EXIT_CODE=$(docker inspect -f '{{.State.ExitCode}}' $CONTAINER_ID)
       - |
         if [ $EXIT_CODE -eq 0 ]; then
           echo "Tests passed!"
         else
           echo "Tests failed with exit code: $EXIT_CODE"
           exit 1
         fi
   

2. 确认正确访问URL
   从docker-compose配置来看，kafkarest与karate处于同一个网络capy-network中，正确的访问URL就是http://kafkarest:38082，无需修改地址。之前的连接问题并非URL错误，而是服务未就绪导致。

3. 可选优化（非必需）
   可以使用wait-for-it等工具增强容器间的依赖检查，确保kafkarest依赖的schema-registry、kafka等服务完全就绪后再启动kafkarest，但流水线中添加轮询等待是最直接的解决方案。

内容的提问来源于stack exchange，提问作者Murrchalkina