容器化Flask+Celery+Redis应用启动Celery Worker时遇Redis属性错误

问题背景 #

尝试以容器化方式运行集成Celery的Flask应用执行后台任务，基于mambaorg/micromamba:latest镜像构建，多次调整配置后Celery Worker始终无法启动。

环境配置 #

依赖清单（environment.yml） #

name: base
channels:
  - conda-forge
  - anaconda
dependencies:
  - python=3.10.6
  - mne=1.1.1
  - mne-features=0.2.1
  - ray-default
  - flask
  - celery
  - redis

Docker Compose配置 #

version: '2.4'

services:
  redis-service:
    image: redis:latest
    container_name: redis-service

  celery-service:
    image: myapp:1.0.0
    container_name: celery-service
    environment:
      - FLASK_APP=run
    volumes:
      - .:/app/
    depends_on:
      - redis-service
    command: celery -A run.celery worker --loglevel=info

  genfeatsbs:
    image: myapp:1.0.0
    container_name: myapp
    environment:
      - FLASK_APP=run
      - FLASK_DEBUG=1
    volumes:
      - .:/app/ # 包含源码和启动脚本run.py
    ports:
      - '5000:5000'
    depends_on:
      - redis-service
    command: flask run -h 0.0.0.0

Flask应用初始化（init.py） #

from celery import Celery, Task
from flask import Flask

from app.routes.execution_blueprint import execution_blueprint

def make_celery(app: Flask) -> Celery:
    class FlaskTask(Task):
        def __call__(self, *args: object, **kwargs: object) -> object:
            with app.app_context():
                return self.run(*args, **kwargs)
        
    celery = Celery(app.name)
    celery.conf.update(app.config["CELERY"])
    celery.Task = FlaskTask
    
    return celery

def create_app(config_file) -> Flask:
    app = Flask(__name__)
    app.config.from_object(config_file)
    
    celery = make_celery(app)
    celery.set_default()
    
    app.register_blueprint(execution_blueprint)
    
    return app, celery

Celery配置 #

CELERY = {
    'broker_url': 'redis://redis-service',
    'result_backend': 'redis://redis-service'
}

启动脚本（run.py） #

from myapp import create_app

app, celery = create_app('app.config.Config')
app.app_context().push()

报错信息 #

Traceback (most recent call last):
  File "/opt/conda/bin/celery", line 10, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/celery/__main__.py", line 15, in main
    sys.exit(_main())
  File "/opt/conda/lib/python3.10/site-packages/celery/bin/celery.py", line 235, in main
    return celery(auto_envvar_prefix="CELERY")
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/celery/bin/base.py", line 134, in caller
    return f(ctx, *args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/celery/bin/worker.py", line 348, in worker
    worker = app.Worker(
  File "/opt/conda/lib/python3.10/site-packages/celery/worker/worker.py", line 98, in __init__
    self.setup_instance(**self.prepare_args(**kwargs))
  File "/opt/conda/lib/python3.10/site-packages/celery/worker/worker.py", line 119, in setup_instance
    self._conninfo = self.app.connection_for_read()
  File "/opt/conda/lib/python3.10/site-packages/celery/app/base.py", line 818, in connection_for_read
    return self._connection(url or self.conf.broker_read_url, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/celery/app/base.py", line 877, in _connection
    return self.amqp.Connection(
  File "/opt/conda/lib/python3.10/site-packages/kombu/connection.py", line 201, in __init__
    if not get_transport_cls(transport).can_parse_url:
  File "/opt/conda/lib/python3.10/site-packages/kombu/transport/__init__.py", line 90, in get_transport_cls
    _transport_cache[transport] = resolve_transport(transport)
  File "/opt/conda/lib/python3.10/site-packages/kombu/transport/__init__.py", line 75, in resolve_transport
    return symbol_by_name(transport)
  File "/opt/conda/lib/python3.10/site-packages/kombu/utils/imports.py", line 59, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/opt/conda/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/conda/lib/python3.10/site-packages/kombu/transport/redis.py", line 266, in <module>
    class PrefixedStrictRedis(GlobalKeyPrefixMixin, redis.Redis):
AttributeError: 'NoneType' object has no attribute 'Redis'

问题根源 #

报错核心是AttributeError: 'NoneType' object has no attribute 'Redis'，说明Kombu（Celery依赖的消息库）尝试导入redis.Redis时，redis模块为None。原因是：

• Conda仓库中的redis包是Redis服务器二进制文件，并非Python连接Redis所需的客户端库。Celery需要的Python客户端库在Conda中名为redis-py，或通过pip安装redis包。

解决方案 #

1. 修正依赖清单 #

将environment.yml中的redis替换为redis-py，确保安装的是Python Redis客户端：

name: base
channels:
  - conda-forge
  - anaconda
dependencies:
  - python=3.10.6
  - mne=1.1.1
  - mne-features=0.2.1
  - ray-default
  - flask
  - celery
  - redis-py  # 替换原有的redis

或者改用pip安装客户端（如果Conda源存在版本兼容问题）：

name: base
channels:
  - conda-forge
  - anaconda
dependencies:
  - python=3.10.6
  - mne=1.1.1
  - mne-features=0.2.1
  - ray-default
  - flask
  - celery
  - pip:
    - redis  # 通过pip安装Python Redis客户端

2. 添加Redis服务健康检查 #

Docker Compose的depends_on仅保证启动顺序，不确保Redis服务就绪。为redis-service添加健康检查，确保Celery Worker在Redis完全启动后再启动：

redis-service:
  image: redis:latest
  container_name: redis-service
  healthcheck:
    test: ["CMD", "redis-cli", "ping"]
    interval: 5s
    timeout: 3s
    retries: 5

celery-service:
  # ... 其他配置不变
  depends_on:
    redis-service:
      condition: service_healthy

3. 明确Celery Redis连接配置 #

在Celery配置中明确端口和数据库索引，避免默认值可能的歧义：

CELERY = {
    'broker_url': 'redis://redis-service:6379/0',
    'result_backend': 'redis://redis-service:6379/0'
}

4. 重新构建镜像并启动服务 #

执行以下命令重新构建应用镜像并启动所有服务：

docker-compose build
docker-compose up -d

验证 #

启动后查看Celery Worker日志：

docker logs celery-service

如果看到类似[INFO] Connected to redis://redis-service:6379/0的日志，说明Worker已成功连接Redis并启动。

内容的提问来源于stack exchange，提问作者Armando Canales Lima