Docker Compose detached模式下Airflow容器无法访问Web服务
Docker 部署Airflow 故障描述
- 部署架构:通过Docker Compose在容器内运行Airflow,整套系统包含两个独立容器:运行Postgres的容器、运行Airflow webserver的容器
- 异常表现:以detached模式(添加
-d参数)启动容器时,无法连接Web服务,访问http://127.0.0.1:8080无任何响应;启动命令示例为docker-compose --env-file .env up -d - 基础排查状态:容器处于正常运行状态,可通过
docker logs airflow_webserver查看日志,也可exec进入容器内部排查 - 对照验证结果:不添加
-d参数、以前台模式启动容器时,服务运行完全正常,访问127.0.0.1:8080可正常使用应用 - 环境差异:该问题仅在搭载Ubuntu系统的EC2实例上运行时出现,在本地Mac电脑上运行相同配置无任何问题
相关配置文件
Docker Compose 配置
version: '3.7' services: postgres: container_name: airflow_pg image: postgres:9.6 env_file: - ../.env ports: - "5432:5432" volumes: - "${LOCALDATABASE}:/var/lib/postgresql/data/pgdata:Z" - ../airflow/logs:/home/airflow/logs:z logging: options: max-size: 10m max-file: "3" webserver: container_name: airflow_webserver user: "${USERID}:1000" build: context: ../ dockerfile: airflow/DockerAirflow args: AIRFLOW_UID: "1000" AIRFLOW_GID: "1000" restart: on-failure env_file: - ../.env depends_on: - postgres logging: options: max-size: 10m max-file: "3" volumes: - ../airflow/dags:/opt/airflow/dags:Z - ../airflow/logs:/opt/airflow/logs:Z - ../app:/opt/airflow/app:Z - "${SSHKEYPATH}:/var/ssh/cbc-pwa-us-east-2-bastion.pem:Z" ports: - "8080:8080" entrypoint: /opt/airflow/start_airflow.sh healthcheck: test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"] interval: 30s timeout: 30s retries: 3
Dockerfile 配置
FROM apache/airflow:master-python3.8 USER root RUN curl -L https://s3.amazonaws.com/redshift-downloads/drivers/odbc/1.4.17.1000/AmazonRedshiftODBC-64-bit-1.4.17.1000-1.x86_64.deb > ~/redshift.deb RUN dpkg -i ~/redshift.deb RUN apt update -y && sudo apt install build-essential -y RUN sudo apt install -y procps RUN usermod -u 1000 airflow RUN groupmod -g 1000 airflow RUN chown airflow:airflow /opt/airflow COPY ./airflow/start_airflow.sh /opt/airflow/start_airflow.sh RUN chmod a+x /opt/airflow/start_airflow.sh USER 1000 ADD ./requirements.txt requirements.txt RUN pip install convertdate lunarcalendar holidays tqdm pystan RUN pip install -r requirements.txt
补充说明
- 配置中对用户ID、用户组做了特殊调整:此前曾出现Airflow镜像默认使用UID 50000的权限异常,由于构建ARG参数未生效,因此做了直接的手动配置,该部分调整目前运行正常
- 暂未定位到故障原因,不确定用户组相关配置调整是否为问题诱因,需要对应的排查方向指导
内容的提问来源于stack exchange,提问作者Craig
相关产品推荐
相关产品推荐

