Docker Compose部署Airflow集成MLflow Model Registry报错求助

我使用Docker Compose集成Airflow与MLflow，流水线运行正常，MLflow可正常追踪参数与指标，但尝试查看已注册模型时出现以下错误：

INVALID_PARAMETER_VALUE: Model registry functionality is unavailable; got unsupported URI './mlruns' for model registry data storage. Supported URI schemes are: ['postgresql', 'mysql', 'sqlite', 'mssql']. See https://www.mlflow.org/docs/latest/tracking.html#storage for how to run an MLflow server against one of the supported backend storage locations.

我的Docker Compose配置如下：

....
services:

  mlflow:
    build:
      dockerfile: docker_mlflow/Dockerfile
    ports:
      - 600:600
      
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: airflow
      POSTGRES_PASSWORD: airflow
      POSTGRES_DB: airflow
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "airflow"]
      interval: 10s
      retries: 5
      start_period: 5s
    restart: always

  redis:
    image: redis:latest
    expose:
      - 6379
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 30s
      retries: 50
      start_period: 30s
    restart: always

  airflow-webserver:
    <<: *airflow-common
    command: webserver
    ports:
      - "8080:8080"
    healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    restart: always
    depends_on:
      <<: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-scheduler:
    <<: *airflow-common
    command: scheduler
    healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:8974/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    restart: always
    depends_on:
      <<: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-worker:
    <<: *airflow-common
    command: celery worker
    healthcheck:
      # yamllint disable rule:line-length
      test:
        - "CMD-SHELL"
        - 'celery --app airflow.providers.celery.executors.celery_executor.app inspect ping -d "celery@$${HOSTNAME}" || celery --app airflow.executors.celery_executor.app inspect ping -d "celery@$${HOSTNAME}"'
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    environment:
      <<: *airflow-common-env
      # Required to handle warm shutdown of the celery workers properly
      # See https://airflow.apache.org/docs/docker-stack/entrypoint.html#signal-propagation
      DUMB_INIT_SETSID: "0"
    restart: always
    depends_on:
      <<: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-triggerer:
    <<: *airflow-common
    command: triggerer
    healthcheck:
      test: ["CMD-SHELL", 'airflow jobs check --job-type TriggererJob --hostname "$${HOSTNAME}"']
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    restart: always
    depends_on:
      <<: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-init:
    <<: *airflow-common
    entrypoint: /bin/bash
    # yamllint disable rule:line-length
    command:
      ( SOME STUFF )

  airflow-cli:
    <<: *airflow-common
    profiles:
      - debug
    environment:
      <<: *airflow-common-env
      CONNECTION_CHECK_MAX_COUNT: "0"
    # Workaround for entrypoint issue. See: https://github.com/apache/airflow/issues/16252
    command:
      - bash
      - -c
      - airflow

  flower:
    <<: *airflow-common
    command: celery flower
    profiles:
      - flower
    ports:
      - "5555:5555"
    healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:5555/"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
    restart: always
    depends_on:
      <<: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

volumes:
  postgres-db-volume:

已尝试配置网络桥接和卷，问题仍未解决。

MLflow模型注册表依赖关系型数据库存储元数据，当前MLflow默认使用本地文件系统（./mlruns），不支持模型注册表功能。可以复用现有Postgres服务来存储MLflow的模型注册表元数据，具体步骤如下：

修改Docker Compose中的MLflow服务配置 #

更新mlflow服务的环境变量、卷和依赖，指定Postgres作为后端存储：

mlflow:
  build:
    dockerfile: docker_mlflow/Dockerfile
  ports:
    - 600:600
  environment:
    # 指定追踪数据存储到Postgres
    - MLFLOW_TRACKING_URI=postgresql://airflow:airflow@postgres:5432/airflow
    # 指定模型注册表元数据存储到Postgres
    - MLFLOW_REGISTRY_URI=postgresql://airflow:airflow@postgres:5432/airflow
    # 配置artifact存储路径（本地卷）
    - MLFLOW_ARTIFACT_ROOT=/mlruns/artifacts
  volumes:
    # 挂载本地卷存储artifact
    - mlflow-artifacts:/mlruns/artifacts
  depends_on:
    # 等待Postgres健康后启动MLflow
    postgres:
      condition: service_healthy

补充MLflow镜像的PostgreSQL驱动 #

在docker_mlflow/Dockerfile中添加PostgreSQL依赖安装：

RUN pip install mlflow psycopg2-binary

更新Airflow任务中的MLflow配置 #

在Airflow DAG的任务代码中，设置正确的MLflow追踪地址：

import mlflow

# 指向MLflow服务地址
mlflow.set_tracking_uri("http://mlflow:600")
# 注册表元数据存储地址与追踪地址一致即可
mlflow.set_registry_uri("postgresql://airflow:airflow@postgres:5432/airflow")

添加MLflow artifact卷 #

在Docker Compose的volumes部分新增：

volumes:
  postgres-db-volume:
  mlflow-artifacts:

关键说明 #

• MLFLOW_TRACKING_URI：指定MLflow存储实验、参数、指标的后端数据库
• MLFLOW_REGISTRY_URI：指定模型注册表的元数据存储位置，必须是支持的关系型数据库
• MLFLOW_ARTIFACT_ROOT：用于存储模型文件、数据集等artifact，可使用本地卷或云存储（如S3）

内容的提问来源于stack exchange，提问作者electricz