Django Celery 5.3.1中time_limit与soft_time_limit不生效问题求助
Celery 5.3.1 time_limit/soft_time_limit 不生效问题
在Django应用中使用Celery 5.3.1的time_limit和soft_time_limit功能时,遇到超时限制完全不生效的问题:执行包含time.sleep(5)的任务始终成功结束,既不触发SoftTimeLimitExceeded异常,也不被强制终止。
Celery配置代码
import os from celery import Celery # Set the default Django settings module for the 'celery' program. os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'server.settings') app = Celery('server') # Using a string here means the worker doesn't have to serialize # the configuration object to child processes. # - namespace='CELERY' means all celery-related configuration keys # should have a `CELERY_` prefix. app.config_from_object('django.conf:settings', namespace='CELERY') app.conf.update( task_soft_time_limit=1, task_time_limit=1, ) # Load task modules from all registered Django apps. app.autodiscover_tasks() @app.task(bind=True, ignore_result=True) def debug_task(self): print(f'Request: {self.request!r}')
任务代码
from celery import shared_task from celery.exceptions import SoftTimeLimitExceeded from celery.signals import task_success, task_failure, task_prerun import time from server.celery import app @shared_task def scrape(domain: str, route: str = None): try: time.sleep(5) except SoftTimeLimitExceeded: print("Time Limit") @task_prerun.connect(sender=scrape) def scrape_prerun_notifier(task_id=None, task=None, *args, **kwargs): print("From task_prerun_notifier ==> Running just before add() executes") @task_success.connect(sender=scrape) def scrape_success_notifier(sender=None, result=None, **kwargs): print("Success") task_id = sender.request.id @task_failure.connect(sender=scrape) def scrape_failure_notifier(task_id=None, exception=None, *args, **kwargs): print("Failure")
已尝试的配置方式
- 将任务装饰器改为
@app.task(soft_time_limit=1, time_limit=1) - 在Django设置中配置
CELERYD_TASK_SOFT_TIME_LIMIT = 1或CELERY_TASK_SOFT_TIME_LIMIT = 1
启动命令
使用以下命令启动Celery Worker:
celery -A server worker -l info --pool=threadscelery -A server worker -l info --pool=solo
尚未尝试prefork池,不清楚如何为其启用日志。请问配置中存在什么问题导致超时限制不生效?
内容的提问来源于stack exchange,提问作者Johanes Lee
相关产品推荐
相关产品推荐

