无法创建第二个Celery队列的问题求助
无法创建第二个Celery队列的问题求助
我想创建两个Celery队列来区分不同类型的任务,但目前遇到了问题——只有默认的celery队列能被正常检测到,我配置的第二个队列celery:1始终看不到。下面是我的详细配置、任务定义和问题现象:
我的Celery配置
这是项目中celery.py的配置代码,我原本期望这段配置能生成celery和celery:1两个队列:
# celery.py import os from celery import Celery from core_app.settings import INSTALLED_APPS # this code copied from manage.py # set the default Django settings module for the 'celery' app. os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core_app.settings') app = Celery("my_app") # To start scheduling tasks based on priorities # you need to configure queue_order_strategy transport option. # ['celery', 'celery:1',] - two queues, the highest priority queue will be named celery app.conf.broker_transport_options = { 'priority_steps': list(range(2)), 'sep': ':', 'queue_order_strategy': 'priority', } # read config from Django settings, the CELERY namespace would make celery # config keys has `CELERY` prefix app.config_from_object('django.conf:settings', namespace='CELERY') # load tasks.py in django apps app.autodiscover_tasks(lambda: INSTALLED_APPS)
任务定义与调用
我分别给两类任务指定了不同的队列,调用方式如下:
@shared_task(queue="celery", soft_time_limit=600, time_limit=650) def check_priority_urls(parsing_result_ids: List[int]): check_urls(parsing_result_ids) @shared_task(queue="celery:1", soft_time_limit=600, time_limit=650) def check_common_urls(parsing_result_ids: List[int]): check_urls(parsing_result_ids) # running task check_priority_urls.delay(parsing_results_ids)
问题现象
我执行命令查看活跃队列时,只返回了celery这一个队列:
celery -A core_app inspect active_queues -> celery@d1a287d1d3b1: OK * {'name': 'celery', 'exchange': {'name': 'celery', 'type': 'direct', 'arguments': None, 'durable': True, 'passive': False, 'auto_delete': False, 'delivery_mode': None, 'no_declare': False}, 'routing_key': 'celery', 'queue_arguments': None, 'binding_arguments': None, 'consumer_arguments': None, 'durable': True, 'exclusive': False, 'auto_delete': False, 'no_ack': False, 'alias': None, 'bindings': [], 'no_declare': None, 'expires': None, 'message_ttl': None, 'max_length': None, 'max_length_bytes': None, 'max_priority': None} 1 node online.
同时在Celery Flower监控界面中,也只能看到一个队列:
Docker Compose中的Celery服务配置
这是我在docker-compose.yml中定义的Celery服务:
celery: build: ./project command: celery -A core_app worker --loglevel=info --concurrency=15 --max-memory-per-child=1000000 volumes: - ./project:/usr/src/app - ./project/media:/project/media - ./project/logs:/project/logs env_file: - .env environment: # environment variables declared in the environment section override env_file - DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1] - CELERY_BROKER=redis://redis:6379/0 - CELERY_BACKEND=redis://redis:6379/0 depends_on: - django - redis
备注:内容来源于stack exchange,提问作者mascai
相关产品推荐
相关产品推荐

