You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

Celery+AWS SQS环境下设置整数retry_backoff导致重试混乱、任务激增问题求助

Celery+AWS SQS环境下设置整数retry_backoff导致重试混乱、任务激增问题求助

大家好,我目前在使用Celery搭配AWS SQS处理异步任务,最近碰到了一个棘手的问题,想向各位大佬求助。

我的任务定义代码如下:

@app.task(
    autoretry_for=(Exception,),
    max_retries=5,
    retry_backoff=True,
    retry_jitter=False,
    acks_late=True,
)
@onfailure_reject(non_traced_exceptions=NON_TRACED_EXCEPTIONS)
def send_order_update_event_task(order_id, data):
    .........

但当我把retry_backoff参数设置为整数值时,重试模式彻底混乱了,任务数量直接失控。

相关错误日志如下:

2024-12-10 05:16:10  
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [53285c923f-79232a3856]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [1052f09663-c19b42589a]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [dd021828dd-4f6b8ae6f8]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [116bef9273-e4dbfb526b]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [913697ae7e-d4f65d45a5]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [d99e889882-a76718b549]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [d99e889882-30bac3e515]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [d7f01e5b4f-edfa22355f]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [8ba15966ae-2266247e56]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [738688f34d-34067ca58b]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [c790586783-b363d38520]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [6231986f4c-7696b7cf47]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10
2024-12-10 05:16:10 
ERROR [1b810665-c0b1-4527-8cd9-c142f67d6605] [e020ded4ca-f11c933d87]  tasks.order_request_task -  [ send_order_update_event_task] Exception for order: 700711926: Order absent 700711926, retry_count: 10

我在代码里打印了每次重试的计数,结果发现同一个重试计数对应了大量重复任务——比如重试计数1有20个任务,计数2有40个,完全超出了预期。

补充下环境信息:所有任务都使用特定队列celery-requests-primary,并且运行在名为celery-requests-primary的K8s部署中,该部署包含多个Pod。

想请教各位,这种情况可能是什么原因导致的?如果需要进一步排查,我还需要提供哪些信息?

备注:内容来源于stack exchange,提问作者Dev

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.04.14 17:34:49