Celery无法通过boto3凭证连接SQS，报InvalidClientTokenId错误求助

问题场景 #

配置Celery与SQS集成时触发InvalidClientTokenId错误，提示请求中的安全令牌无效。

使用版本 #

• celery[sqs]==5.3.6
• Kombu==5.3.7

配置代码 #

# Celery configuration
SQS_QUEUE_URL = f'https://sqs.us-east-2.amazonaws.com/{settings.aws_account}/{settings.celery_worker_queue}' 
result_backend = f'db+postgresql://{settings.pg_db_config.user}:{settings.pg_db_config.pwd}@{settings.pg_db_config.host}:5432/{settings.pg_db_config.db_name}'
pg_db_conn_string = f"postgresql://{settings.pg_db_config.user}:{settings.pg_db_config.pwd}@{settings.pg_db_config.host}:{settings.pg_db_config.port}/{settings.pg_db_config.db_name}"

session = boto3.Session(profile_name='ar-se-sbx')

app = Celery(
    'worker',
    broker_url=f"sqs://",
    backend=result_backend,
    task_default_queue=settings.celery_worker_queue,
    broker_connection_retry_on_startup=True,
    broker_transport_options={
        "region": "us-east-2",
        'predefined_queues': {
            'celery-worker-1':{
                'url': SQS_QUEUE_URL,
                'access_key_id': session.get_credentials().access_key,
                'secret_access_key': session.get_credentials().secret_key,
            }
        }
    },
    task_create_missing_queues=False,
)

错误日志 #

--------------- celery@XXXXXv5.3.6 (emerald-rush)
--- ***** ----- 
-- ******* ---- Windows-10-10.0.22631-SP0 2024-09-21 17:40:48
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         worker:0x1f0861a8370
- ** ---------- .> transport:   sqs://localhost//
- ** ---------- .> results:     postgresql://root:**@127.0.0.1:5432/postgres
- *** --- * --- .> concurrency: 12 (solo)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery-worker-1  exchange=celery-worker-1(direct) key=celery-worker-1


[tasks]
  . LANG_GUARD

[2024-09-21 17:40:48,995: INFO/MainProcess] Connected to sqs://localhost//
[2024-09-21 17:40:49,832: CRITICAL/MainProcess] Unrecoverable error: ClientError('An error occurred (InvalidClientTokenId) when calling the GetQueueAttributes operation: The security token included in the request is invalid.')

解决步骤 #

1. 验证AWS凭证有效性 #

• 检查本地AWS凭证文件（Windows默认路径：C:\Users\<用户名>\.aws\credentials）中ar-se-sbx配置段的aws_access_key_id和aws_secret_access_key是否拼写正确、未过期。
• 用AWS CLI测试凭证权限：

  aws sqs get-queue-attributes --queue-url "你的SQS队列URL" --profile ar-se-sbx
  

  若同样报错，说明凭证本身无效，需重新生成或更新。

2. 调整Celery凭证配置位置 #

当前将凭证放在predefined_queues内，Celery的SQS传输可能无法正确读取，建议改为全局配置：

方式一：在Broker URL中嵌入凭证

app = Celery(
    'worker',
    broker_url=f"sqs://{session.get_credentials().access_key}:{session.get_credentials().secret_key}@",
    backend=result_backend,
    task_default_queue=settings.celery_worker_queue,
    broker_connection_retry_on_startup=True,
    broker_transport_options={
        "region": "us-east-2",
        'predefined_queues': {
            'celery-worker-1':{
                'url': SQS_QUEUE_URL,
            }
        }
    },
    task_create_missing_queues=False,
)

方式二：在全局Transport选项中设置凭证

app = Celery(
    'worker',
    broker_url=f"sqs://",
    backend=result_backend,
    task_default_queue=settings.celery_worker_queue,
    broker_connection_retry_on_startup=True,
    broker_transport_options={
        "region": "us-east-2",
        'access_key_id': session.get_credentials().access_key,
        'secret_access_key': session.get_credentials().secret_key,
        'predefined_queues': {
            'celery-worker-1':{
                'url': SQS_QUEUE_URL,
            }
        }
    },
    task_create_missing_queues=False,
)

3. 确认boto3凭证加载逻辑 #

添加打印语句验证boto3是否正确读取凭证：

print("Access Key:", session.get_credentials().access_key)
print("Secret Key:", session.get_credentials().secret_key)

若输出为空或不正确，检查凭证文件权限、路径是否正确，或环境变量是否覆盖了配置文件。

4. 检查IAM用户权限 #

确保该AWS凭证对应的IAM用户拥有SQS操作权限，至少包含sqs:GetQueueAttributes、sqs:SendMessage、sqs:ReceiveMessage等权限，可通过IAM控制台查看用户的权限策略。

内容的提问来源于stack exchange，提问作者Bells2025