Celery Worker接收Django任务但未执行问题求助

环境配置：

• Django开发服务器：127.0.0.1:8000
• Celery Worker：v5.3.1
• Redis：Docker容器运行在redis://127.0.0.1:6379
• 邮件后端：EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

预期行为：注册测试用户时，控制台输出邮件内容
实际行为：无邮件输出，Celery任务被标记为"Processed"但未转为成功/失败状态

tasks.py 代码 #

@shared_task
def send_welcome_email(user_email):
    print("send welcome email started")
    subject = 'Welcome to My Website'
    message = 'Thank you for registering at My Website!'
    from_email = 'noreply@mywebsite.com'
    recipient_list = [user_email]
    print(f"Sending email to {user_email}")

    send_mail(subject, message, from_email, recipient_list)
    print("Email sent")

已完成的配置 #

• 已将Celery App添加到项目__init__.py
• 配置了Celery的Broker URL和结果后端
• 创建了项目根目录的celery.py文件
• 确认Celery与Redis连接正常

测试现象 #

• 通过Flower查看，Celery Worker已接收任务，状态显示"Processed"但无后续状态变更
• 项目Shell中调用send_welcome_email.delay('test@gmail.com')无输出；直接调用（不带.delay()）时控制台有邮件输出
• Celery无过载，连接正常

Celery 日志 #

-------------- celery@POLARBY v5.3.1 (emerald-rush)
--- ***** ----- 
-- ******* ---- Windows-10-10.0.22621-SP0 2023-09-06 01:06:38
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         Kemestra:0x2158f8da190
- ** ---------- .> transport:   redis://127.0.0.1:6379/0
- ** ---------- .> results:     redis://127.0.0.1:6379/0
- *** --- * --- .> concurrency: 12 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . account.tasks.send_welcome_email
  . account.tasks.simple_task

[2023-09-06 01:06:38,480: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0

[2023-09-06 01:06:38,488: INFO/MainProcess] mingle: searching for neighbors
[2023-09-06 01:06:38,803: INFO/SpawnPoolWorker-3] child process 15324 calling self.run()
[2023-09-06 01:06:38,805: INFO/SpawnPoolWorker-2] child process 5212 calling self.run()
[2023-09-06 01:06:38,812: INFO/SpawnPoolWorker-8] child process 16680 calling self.run()
[2023-09-06 01:06:38,813: INFO/SpawnPoolWorker-4] child process 14388 calling self.run()
[2023-09-06 01:06:38,814: INFO/SpawnPoolWorker-1] child process 1040 calling self.run()
[2023-09-06 01:06:38,815: INFO/SpawnPoolWorker-9] child process 352 calling self.run()
[2023-09-06 01:06:38,817: INFO/SpawnPoolWorker-7] child process 15132 calling self.run()
[2023-09-06 01:06:38,830: INFO/SpawnPoolWorker-6] child process 8304 calling self.run()
[2023-09-06 01:06:38,835: INFO/SpawnPoolWorker-10] child process 6444 calling self.run()
[2023-09-06 01:06:38,843: INFO/SpawnPoolWorker-5] child process 17908 calling self.run()
[2023-09-06 01:06:38,850: INFO/SpawnPoolWorker-11] child process 9556 calling self.run()
[2023-09-06 01:06:38,875: INFO/SpawnPoolWorker-12] child process 11840 calling self.run()
[2023-09-06 01:06:39,474: INFO/SpawnPoolWorker-13] child process 5108 calling self.run()
[2023-09-06 01:06:39,474: INFO/SpawnPoolWorker-14] child process 8028 calling self.run()
[2023-09-06 01:06:39,519: INFO/MainProcess] mingle: all alone
[2023-09-06 01:06:39,540: INFO/MainProcess] celery@POLARBY ready.
[2023-09-06 01:06:39,966: INFO/SpawnPoolWorker-15] child process 17104 calling self.run()
[2023-09-06 01:07:10,515: INFO/MainProcess] Task account.tasks.send_welcome_email[40528b4a-46ca-45b1-8345-642883695b0a] received
[2023-09-06 01:07:10,954: INFO/SpawnPoolWorker-18] child process 6648 calling self.run()
[2023-09-06 01:07:10,955: INFO/SpawnPoolWorker-17] child process 18112 calling self.run()
[2023-09-06 01:07:10,955: INFO/SpawnPoolWorker-16] child process 15540 calling self.run()

排查方向与解决方案 #

1. Windows系统下Prefork模式兼容性问题 #

Celery的prefork并发模式在Windows上存在兼容性问题，任务可能卡住无法执行。

• 解决：启动Celery时指定单进程模式：

celery -A your_project_name worker -l info --pool=solo

2. Django配置未在Celery Worker中加载 #

Celery Worker进程可能未正确加载Django的邮件配置，导致send_mail无法正常工作。

• 解决：在tasks.py开头添加Django初始化代码：

import os
import django
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'your_project_name.settings')
django.setup()

from celery import shared_task
from django.core.mail import send_mail

3. 日志输出位置问题 #

Celery Worker的print输出不会显示在Django开发服务器的控制台，而是在Celery Worker的终端窗口中。

• 验证：查看启动Celery Worker的终端界面。

4. 进程卡住或资源不足 #

从日志看任务已接收但无执行日志，可能是Worker进程卡住或资源不足。

• 解决：
  • 重启Celery Worker和Redis容器
  • 检查Redis容器运行状态：docker ps查看容器状态，docker logs <redis_container_id>检查日志
  • 减少Celery并发数：启动时添加--concurrency=1参数

5. 结果后端配置问题 #

任务状态显示"Processed"但无成功/失败标记，可能是结果后端未正确配置。

• 验证：在Shell中调用result.get()查看是否抛出异常，result.state查看任务状态
• 确保celery.py中结果后端配置正确：

app.conf.result_backend = 'redis://127.0.0.1:6379/0'

内容的提问来源于stack exchange，提问作者notmclovinit