首次配置Django-q使用Amazon SQS作为消息代理的技术咨询

Hey there! Let's get your Django-Q configured with Amazon SQS smoothly—since you already have AWS experience with S3/CloudFront, half the battle is already won. Here's a step-by-step guide tailored to your specific versions (Python 2.7.11, Django 1.11.5, Django-q 0.9.2):

1. Add Django-Q to Your Installed Apps #

First, make sure django_q is in your INSTALLED_APPS in settings.py—this lets Django recognize the app:

INSTALLED_APPS = [
    # Your existing apps (like django.contrib.admin, etc.)
    'django_q',
]

2. Configure the Q_CLUSTER for SQS #

Next, add the Q_CLUSTER configuration block to settings.py. This tells Django-Q to use SQS as the broker, and includes all the necessary AWS details.

Important: Avoid hardcoding your AWS credentials—use environment variables instead (you’re probably already doing this for S3, so it’s familiar territory). Here’s the setup:

import os

Q_CLUSTER = {
    'name': 'MyDjangoQSQSCluster',  # A unique name for your cluster
    'workers': 4,  # Adjust based on your server's CPU/memory (start with 2-4)
    'timeout': 30,  # Max time a task can run before timing out
    'retry': 60,  # Seconds before retrying a failed task
    'queue_limit': 50,  # Max number of tasks allowed in the queue
    'bulk': 10,  # Number of tasks to fetch at once from SQS
    'orm': 'default',  # Use your default Django DB to store task results
    # SQS Broker Configuration
    'broker': 'django_q.brokers.sqs.SQS',
    'broker_options': {
        'aws_access_key_id': os.environ.get('AWS_SQS_ACCESS_KEY_ID'),
        'aws_secret_access_key': os.environ.get('AWS_SQS_SECRET_ACCESS_KEY'),
        'region_name': 'us-east-1',  # Replace with your SQS queue's region
        'queue_name': 'your-sqs-queue-name',  # Replace with your actual queue name
    }
}

• Double-check that the region_name matches the region where you created your SQS queue.
• The IAM user you created with AmazonSQSFullAccess should work perfectly here—no need to tweak permissions for initial setup.

3. Test the Configuration #

Once your settings are in place, let’s verify everything works:

Step 3.1 Start the Django-Q Worker #

Run this command in your terminal to start the task worker:

python manage.py qcluster

You should see logs indicating the worker is connected to SQS and waiting for tasks.

Step 3.2 Send a Test Task #

Open another terminal, launch the Django shell, and send a simple test task:

python manage.py shell

Then in the shell:

from django_q.tasks import async_task

def test_sqs_task():
    print("Success! SQS task executed.")

# Queue the task
async_task(test_sqs_task)

Check the worker terminal—you should see logs showing the task was received and executed. If you do, your setup is working!

Troubleshooting Common Issues #

If you run into problems, here’s what to check first:

• Credential/Region Mismatch: Ensure your AWS keys are correct, the region matches your SQS queue, and the queue name is spelled exactly as it appears in the AWS console.
• Python 2.7 Compatibility: Your boto3 version (1.5.28) is compatible with Python 2.7, but if you see dependency errors, run pip install -r requirements.txt to ensure all packages are properly installed.
• Queue Access: Confirm your IAM user has access to the specific SQS queue (the AmazonSQSFullAccess policy should cover this, but you can verify in the AWS IAM console if needed).

内容的提问来源于stack exchange，提问作者Ljubisa Livac