Celery/RabbitMQ权限管理——交换机/队列配置适配咨询

Hey there! Since your DevOps team has created a RabbitMQ user without any configuration permissions, we need to adjust your Celery setup to avoid attempts to create or modify queues (which would fail due to the restricted permissions). Here's a step-by-step breakdown to get this working:

Step 1: Coordinate with Your DevOps Team First #

First, make sure your DevOps team completes these two critical tasks:

• Pre-create all required queues: They need to manually create every queue you've defined in your Celery config: default, classifier, clients, bookings, ingestor, and typeahead in the target RabbitMQ vhost.
• Grant targeted permissions: The user needs read and write access to these specific queues (but no configure permissions). A sample rabbitmqctl command they can use (adjust the vhost and username to match your setup):

  rabbitmqctl set_permissions -p / restricted_user "" "^(default|classifier|clients|bookings|ingestor|typeahead)$" "^(default|classifier|clients|bookings|ingestor|typeahead)$"
  

  Let's unpack this command:
  • The empty string "" for the configure parameter ensures the user can't modify any RabbitMQ resources (aligning with DevOps's restriction).
  • The regex patterns for write and read limit the user to only interacting with your specific queues.

Step 2: Update Your Celery Configuration #

Modify your Celery app config to stop trying to create missing queues (since they're already pre-created by DevOps):

For Celery 4.x+ (including 5.x): #

# Disable automatic creation of missing queues
app.conf.task_create_missing_queues = False

# Keep your existing queue declarations (no changes needed here)
app.conf.CELERY_QUEUES = (
    Queue('default', routing_key='default'),
    Queue('classifier', routing_key='classifier'),
    Queue('clients', routing_key='clients'),
    Queue('bookings', routing_key='bookings'),
    Queue('ingestor', routing_key='ingestor'),
    Queue('typeahead', routing_key='typeahead')
)

For Older Celery Versions (pre-4.x): #

Use the legacy setting name instead:

app.conf.CELERY_CREATE_MISSING_QUEUES = False

Step 3: Verify the Setup #

• Start your Celery worker and send test tasks to each queue.
• If you encounter AccessRefused errors, double-check:
  1. DevOps created all queues with exact matching names.
  2. The permissions were applied correctly (verify with rabbitmqctl list_permissions -p /).
  3. Your Celery routing keys match the ones used when DevOps created the queues (your current config looks good as long as they used the same routing keys).

This setup ensures Celery only checks for existing queues (instead of trying to create them) and operates within the restricted user's permissions.

内容的提问来源于stack exchange，提问作者Busturdust