Docker化后Celery任务结果无法写入数据库的问题排查

API 探路者

2026-8-25

Django + Celery Docker化后任务结果无法写入SQLite数据库问题排查

问题描述

已完成Django、gunicorn、nginx和Celery的Docker化部署，Celery任务日志显示执行成功，但结果无法存入数据库。未Docker化前功能正常，当前使用SQLite3作为数据库。

配置文件

docker-compose.yml

version: '3.8'

services:
  django_gunicorn:
    volumes:
      - db:/db.sqlite3
      - static:/static
      - media:/media
    env_file:
      - env
    build:
      context: .
    ports: 
      - "8000:8000"
    command: sh -c "python manage.py migrate && python manage.py collectstatic --no-input && gunicorn main.wsgi:application --bind 0.0.0.0:8000"

  nginx:
    build: ./nginx
    volumes:
      - static:/static
      - media:/media
    ports:
      - "80:80"
    depends_on:
      - django_gunicorn

  rabbitmq3:
    image: rabbitmq:3-alpine
    ports:
      - 5672:5672

  celery:
    restart: always
    build:
      context: .
    command: celery -A main worker -P eventlet -c 100 -l INFO
    env_file:
      - env
    depends_on:
      - rabbitmq3
      - django_gunicorn
    volumes:
      - db:/db.sqlite3

volumes:
  db:
  static:
  media:

Dockerfile

FROM python:3.10.5-alpine

ENV PYTHONUNBEFFERED = 1 

RUN pip install --upgrade pip

COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY ./src /app

WORKDIR /app

celery.py

import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'main.settings')

app = Celery('main', backend='django-db')

app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

settings.py（补充）

from pathlib import Path
import os

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.getenv('SECRET_KEY')
DEBUG = os.getenv('DEBUG')

# SECURITY WARNING: don't run with debug turned on in production!
#DEBUG = True

ALLOWED_HOSTS = ["127.0.0.1"]


#CSRF_COOKIE_SECURE = True

#SESSION_COOKIE_SECURE = True


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_celery_results',
    'django_celery_beat',
    'crispy_forms',
    'crispy_bootstrap5',
    'fontawesomefree',
    'bootstrap_datepicker_plus',
    'scripts.apps.ScriptsConfig',
    'django_cleanup.apps.CleanupConfig',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'main.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / 'templates'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'main.wsgi.application'


# Database
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}


# Password validation
# https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/3.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.2/howto/static-files/

STATIC_ROOT = BASE_DIR / 'static'

STATIC_URL = '/static/'


# Media files (scripts, results)

MEDIA_ROOT = BASE_DIR

MEDIA_URL = 'scripts/'

# Default primary key field type
# https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

# Crispy with Bootstrap

CRISPY_TEMPLATE_PACK = 'bootstrap5'

CRISPY_ALLOWED_TEMPLATE_PACKS = "bootstrap5"

CELERY_BROKER_URL="amqp://guest:guest@rabbitmq3:5672/"

问题分析与解决方案

1. 核心问题：卷挂载路径不匹配

当前配置中，django_gunicorn和celery服务挂载db:/db.sqlite3，但容器内的BASE_DIR是/app（WORKDIR设置为/app，src代码被复制到/app目录），所以SQLite数据库的实际路径是/app/db.sqlite3。两个容器读写的是不同的数据库文件，这是任务结果无法写入的直接原因。

2. 临时修复方案（继续使用SQLite）

修改docker-compose.yml的卷挂载路径

将django_gunicorn和celery的db卷挂载路径改为容器内实际的数据库路径：

services:
  django_gunicorn:
    volumes:
      - db:/app/db.sqlite3
      - static:/static
      - media:/media
    # 其他配置不变

  celery:
    volumes:
      - db:/app/db.sqlite3
    # 其他配置不变

确保Celery容器执行数据库迁移

django_celery_results需要的表由migrate生成，当前只有django容器执行了migrate，可给Celery服务添加migrate步骤：

celery:
  command: sh -c "python manage.py migrate && celery -A main worker -P eventlet -c 100 -l INFO"

验证容器内文件权限

在Dockerfile中添加权限配置，确保/app目录可写入：

RUN mkdir -p /app && chmod -R 755 /app

3. 长期解决方案：切换到PostgreSQL

SQLite是单进程数据库，多容器场景下的文件锁机制容易引发写入异常，若有并发需求，建议切换到PostgreSQL：

在docker-compose.yml中添加PostgreSQL服务

services:
  db:
    image: postgres:15-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}

  django_gunicorn:
    depends_on:
      - db
    # 移除原db卷挂载配置

  celery:
    depends_on:
      - db
    # 移除原db卷挂载配置

volumes:
  postgres_data:
  static:
  media:

修改settings.py的数据库配置

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.getenv('DB_NAME'),
        'USER': os.getenv('DB_USER'),
        'PASSWORD': os.getenv('DB_PASSWORD'),
        'HOST': 'db',
        'PORT': 5432,
    }
}

更新env文件添加数据库变量

DB_NAME=your_db_name
DB_USER=your_db_user
DB_PASSWORD=your_db_password

内容的提问来源于stack exchange，提问作者Balizok