Django环境下如何根据运行环境为Celery加载对应settings文件？

Hey there! Let's tackle how to load the right Django settings for Celery based on your environment (dev/prod/test) when using Supervisor. Here's a straightforward, battle-tested approach:

1. Organize Your Django Settings Structure #

First, make sure your settings are split into environment-specific files. A clean, maintainable structure looks like this:

your_project/
├── your_project/
│   ├── settings/
│   │   ├── __init__.py
│   │   ├── base.py       # Shared core settings (imported by other files)
│   │   ├── development.py # Dev-specific overrides
│   │   ├── production.py  # Prod-specific overrides
│   │   └── testing.py     # Test-specific overrides
│   ├── celery.py
│   └── ...
└── ...

Each environment file extends base.py so you don't repeat shared configuration.

2. Configure Celery to Read Environment Variables #

Update your project's celery.py file to pull the Django settings module from an environment variable. This lets Supervisor pass in the correct value for each environment:

import os
from celery import Celery

# Fallback to development settings if no env var is set (safe for local testing)
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project.settings.development')

app = Celery('your_project')

# Use Django's settings for Celery, with the CELERY namespace to avoid config conflicts
app.config_from_object('django.conf:settings', namespace='CELERY')

# Auto-discover tasks from all installed Django apps
app.autodiscover_tasks()

3. Create Supervisor Configs for Each Environment #

Make a separate Supervisor config file for every environment. This is where you'll explicitly set the DJANGO_SETTINGS_MODULE variable.

Example: Development Environment Config (/etc/supervisor/conf.d/celery-dev.conf) #

[program:celery-dev]
command=/path/to/your/venv/bin/celery -A your_project worker --loglevel=info
directory=/path/to/your/project/root
user=your_system_username
numprocs=1
stdout_logfile=/var/log/celery/celery-dev.log
stderr_logfile=/var/log/celery/celery-dev.err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
# Point to dev settings
environment=DJANGO_SETTINGS_MODULE="your_project.settings.development"

Example: Production Environment Config (/etc/supervisor/conf.d/celery-prod.conf) #

[program:celery-prod]
command=/path/to/your/venv/bin/celery -A your_project worker --loglevel=info --concurrency=4
directory=/path/to/your/project/root
user=your_system_username
numprocs=1
stdout_logfile=/var/log/celery/celery-prod.log
stderr_logfile=/var/log/celery/celery-prod.err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
# Point to prod settings
environment=DJANGO_SETTINGS_MODULE="your_project.settings.production"

Testing Environment Config #

Copy the dev config, rename it to celery-test.conf, and update the environment line to point to your_project.settings.testing.

4. Set Up Logs & Reload Supervisor #

First, ensure the log directory exists to avoid startup errors:

sudo mkdir -p /var/log/celery
sudo chown your_system_username:your_system_username /var/log/celery

Then reload Supervisor to apply the new configs and start the workers:

# Reread all Supervisor configs
sudo supervisorctl reread
# Update Supervisor to register new programs
sudo supervisorctl update

# Start the dev worker (swap with celery-prod/celery-test for other environments)
sudo supervisorctl start celery-dev

Quick Tips #

• If you use .env files for local development, keep using them—but Supervisor doesn't read .env by default, so setting the environment variable directly in the Supervisor config is the most reliable approach for server environments.
• For production, tweak --concurrency to match your server's CPU cores (a good starting point is number of cores + 1).
• To verify the correct settings are loaded, add a debug task that prints settings.__file__ and run it with Celery.

内容的提问来源于stack exchange，提问作者Cody