Docker环境中Celery+AWS SQS任务启动后立即取消问题求助

环境 #

• Django版本：4.2
• Celery版本：5.3.6
• Broker：AWS SQS
• 部署方式：Docker

Celery配置 #

from __future__ import absolute_import, unicode_literals
from celery import Celery, shared_task
from firmwave import settings

app = Celery(
    'firmwave',
    broker=f'sqs://{settings.AWS_SQS_ACCESS_KEY}:{settings.AWS_SQS_SECRET_KEY}@',
)

app.conf.broker_transport_options = {
    'region': settings.AWS_SQS_REGION,
    'predefined_queues': {
        'celery': {
            'url': settings.AWS_QSQ_QUEUE_URL,
            'access_key_id': settings.AWS_SQS_ACCESS_KEY,
            'secret_access_key': settings.AWS_SQS_SECRET_KEY,
        },
    },
    'is_secure': True,
}

app.conf.enable_utc = True
app.conf.broker_connection_retry_on_startup = True
app.autodiscover_tasks()

调试输出 #

[DEBUG/MainProcess] Event request-created.sqs.GetQueueAttributes: calling handler...
[DEBUG/MainProcess] Response body:
b'<?xml version="1.0"?><GetQueueAttributesResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/">**<GetQueueAttributesResult><Attribute><Name>ApproximateNumberOfMessages</Name><Value>155</Value></Attribute></GetQueueAttributesResult>**<ResponseMetadata><RequestId>922533a1-ce80-5018-8f23-6d970e5039f6</RequestId></ResponseMetadata></GetQueueAttributesResponse>'
[2024-02-02 10:06:08,319: DEBUG/MainProcess] Event needs-retry.sqs.GetQueueAttributes: calling handler <botocore.retryhandler.RetryHandler object at 0xffffb3475970>
[DEBUG/MainProcess] Sending http request...
[DEBUG/MainProcess] Certificate path...
[DEBUG/MainProcess] Response headers...
[DEBUG/MainProcess] Response body...
[DEBUG/MainProcess] No retry needed.
[DEBUG/MainProcess] Canceling task consumer...
[DEBUG/MainProcess] Closing consumer channel...

问题描述与已排查项 #

Celery Worker可成功连接AWS SQS，但会取消任务消费者并关闭通道，未处理任何任务。已完成以下排查：

• 确认AWS SQS队列存在且可访问
• 验证AWS凭证正确且权限充足
• 本地非Docker环境（PyCharm）运行正常

怀疑问题与Docker网络配置或Celery适配Docker的设置有关，但不确定具体调整点。求解决思路或类似问题的经验。

内容的提问来源于stack exchange，提问作者Stefan Ionut