Docker部署Django项目配置Celery时出现模块加载错误求助

错误日志 #

celery-worker               | Usage: celery [OPTIONS] COMMAND [ARGS]...
celery-worker               | Try 'celery --help' for help.
celery-worker               | 
celery-worker               | Error: Invalid value for '-A' / '--app': 
celery-worker               | Unable to load celery application.
celery-worker               | While trying to load the module backend.celery_app.app the following error occurred:
celery-worker               | Traceback (most recent call last):
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/celery/bin/celery.py", line 58, in convert
celery-worker               |     return find_app(value)
celery-worker               |            ^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/celery/app/utils.py", line 383, in find_app
celery-worker               |     sym = symbol_by_name(app, imp=imp)
celery-worker               |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/kombu/utils/imports.py", line 59, in symbol_by_name
celery-worker               |     module = imp(module_name, package=package, **kwargs)
celery-worker               |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
celery-worker               |     return imp(module, package=package)
celery-worker               |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
celery-worker               |     return _bootstrap._gcd_import(name[level:], package, level)
celery-worker               |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
celery-worker               |   File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
celery-worker               |   File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
celery-worker               |   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
celery-worker               |   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
celery-worker               |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
celery-worker               |   File "/backend/backend/celery_app.py", line 10, in <module>
celery-worker               |     app.conf.broker_url = settings.CELERY_BROKER_URL
celery-worker               |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
celery-worker               |     self._setup(name)
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
celery-worker               |     self._wrapped = Settings(settings_module)
celery-worker               |                     ^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
celery-worker               |     mod = importlib.import_module(self.SETTINGS_MODULE)
celery-worker               |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
celery-worker               |     return _bootstrap._gcd_import(name[level:], package, level)
celery-worker               |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
celery-worker               |   File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
celery-worker               |   File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
celery-worker               |   File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
celery-worker               |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
celery-worker               |   File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
celery-worker               |   File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
celery-worker               |   File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked
celery-worker               | ModuleNotFoundError: No module named 'core'
celery-worker               | 
celery-worker exited with code 2

项目结构 #

web-for-cyclers/
  backend/
    core/
      __init__.py
      asgi.py
      settings.py
      urls.py
      wsgi.py
     
    static
    venv
    Dockerfile
    manage.py
    requirements.txt
    web.env
    .dockerignore
    .gitignore
    celery_app.py

 
  frontend/
    node_modules/
    public/
    src/
    .dockerignore
    Dockerfile
    package.json
    package-lock.json
    .gitignore
    

  nginx/
    Dockerfile
    nginx.conf
  docker-compose.yml

相关文件内容 #

celery_app.py #

import os

from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')

app = Celery('backend.core')
app.config_from_object('django.conf:settings')
app.conf.broker_url = settings.CELERY_BROKER_URL
app.autodiscover_tasks()

core/settings.py（Broker配置） #

CELERY_BROKER_URL = 'redis://redis:6379/0'

core/init.py #

from celery_app import app as celery_app

__all__ = ('celery_app',)

docker-compose.yml #

services:

  # PostgreSQL
  db:
    image: postgres:14.1-alpine
    restart: always
    env_file:
      - backend/web.env
    ports:
      - '5432:5432'
    volumes:
      - db:/var/lib/postgresql/data

  # Backend Django
  backend:
    build:
      context: ./backend
    command: bash -c "./manage.py collectstatic --noinput && ./manage.py migrate && gunicorn -b 0.0.0.0:8000 core.wsgi:application —log-level debug"
    ports:
      - '8000:8000'
    depends_on:
      - db
    env_file:
      - backend/web.env
    volumes:
      - ./backend/:/backend
      - static_volume:/backend/static
      - media_volume:/backend/media

  # Frontend Next.js
  frontend:
    build:
      context: ./frontend
    command: npm run dev
    depends_on:
      - backend
    volumes:
      - .:/app/frontend
      - /app/node_modules
      - /app/.next
    ports:
      - '3000:3000'

  # Nginx
  nginx:
    build:
      dockerfile: ./Dockerfile
      context: ./nginx
    ports:
      - '80:80'
    volumes:
      - static_volume:/backend/static
      - media_volume:/backend/media
    depends_on:
      - backend
      - frontend
      - db

  # Redis
  redis:
    image: redis:7.0.5-alpine
    container_name: redis

  # Celery Worker 1
  celery-worker:
    restart: always
    build:
      context: ./backend
    entrypoint: celery
    command: -A backend.celery_app.app worker --loglevel=info
    volumes:
      - .:/backend
    container_name: celery-worker
    depends_on:
      - db
      - redis
      - backend
    links:
      - redis
    env_file:
      - backend/web.env

  # Flower
  flower:
    build:
      context: ./backend
    container_name: flower
    command: celery -A backend.celery_app.app flower
    links:
      - redis
    depends_on:
      - redis
      - backend
      - db
    ports:
      - '5555:5555'
    volumes:
      - .:/backend


volumes:
  db:
    driver: local
  react_build:
  static_volume:
    driver: local
  media_volume:
    driver: local

解决方法 #

1. 修正容器工作目录与Celery命令路径 #

问题核心是Celery容器的工作目录不对，导致无法找到core模块。修改docker-compose.yml中的celery-worker和flower服务：

修改celery-worker服务：

celery-worker:
  restart: always
  build:
    context: ./backend
  working_dir: /backend  # 设置工作目录为backend目录
  entrypoint: celery
  command: -A celery_app.app worker --loglevel=info  # 简化模块路径，直接引用celery_app.app
  volumes:
    - ./backend/:/backend
  container_name: celery-worker
  depends_on:
    - db
    - redis
    - backend
  links:
    - redis
  env_file:
    - backend/web.env

修改flower服务：

flower:
  build:
    context: ./backend
  working_dir: /backend  # 设置工作目录为backend目录
  container_name: flower
  command: celery -A celery_app.app flower  # 简化模块路径
  links:
    - redis
  depends_on:
    - redis
    - backend
    - db
  ports:
    - '5555:5555'
  volumes:
    - ./backend/:/backend

2. 确保core目录为Python包 #

检查core目录下的__init__.py文件名是否正确（必须是__init__.py，而非__init.py__），否则Python不会将该目录识别为可导入的包。

3. 可选：添加Python路径到celery_app.py #

如果上述修改后仍有问题，可在celery_app.py开头添加代码，将当前目录加入Python路径：

import os
import sys

# 将当前文件所在目录加入Python路径
sys.path.append(os.path.dirname(os.path.abspath(__file__)))

from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')

app = Celery('backend.core')
app.config_from_object('django.conf:settings')
app.conf.broker_url = settings.CELERY_BROKER_URL
app.autodiscover_tasks()

4. 验证容器内文件挂载 #

执行以下命令，检查容器内是否正确挂载了backend目录及core子目录：

docker-compose run celery-worker ls -la /backend

确认输出中存在core目录。

内容的提问来源于stack exchange，提问作者Mihail Bury