API Gateway请求过多异常(429)排查：Bitbucket流水线报错原因分析

Let's dive into why you're facing this 429 error and how to fix it—this is a common issue when combining Serverless deployments with follow-up API Gateway management scripts, so you're not alone.

Possible Root Causes #

1. AWS API Management Rate Limits Are Being Exceeded #

AWS enforces rate limits on all its service APIs, including the ones used to manage API Gateway (like CreateModel which your script calls). The Serverless deploy command (./node_modules/.bin/serverless deploy -s devme) already sends dozens of API requests to AWS to set up/update Lambda functions, API Gateway resources, IAM roles, and more. When your JS script runs immediately after, it adds more requests to the same short window, pushing you over the rate threshold for API Gateway management operations.

By default, many AWS management APIs have relatively strict rate limits (e.g., CreateModel might be capped at a few requests per second at the account level). These limits are in place to prevent abuse, but they can trigger 429s when consecutive operations flood the service.

2. No Cool-Down Between Serverless Deploy and Script Execution #

Your pipeline runs the Serverless deploy and then the update script back-to-back with no delay. Serverless deployments can take several seconds to complete, and AWS might still be processing some of the background requests from the deployment when your script starts sending more. This continuous burst of requests easily crosses the rate limit threshold.

3. Concurrent Pipeline Executions #

If you have multiple Bitbucket pipelines running the same deployment + script workflow at the same time (e.g., multiple developers pushing changes simultaneously), the combined requests from all these pipelines will multiply the load on AWS's APIs, making 429 errors much more likely.

Fixes and Workarounds #

Implement Exponential Backoff and Retries in Your JS Script #

The most reliable long-term fix is to add retry logic with exponential backoff to your script. AWS SDKs (both v2 and v3) have built-in support for this, which automatically retries 429 errors with increasing wait times between attempts.

For example, with AWS SDK v2:

const AWS = require('aws-sdk');
const apigateway = new AWS.APIGateway({
  maxRetries: 5,
  retryDelayOptions: { base: 1000 } // Start with 1s delay, double each retry
});

// Your CreateModel logic here
apigateway.createModel(params, (err, data) => {
  // Handle response
});

For SDK v3, use the @aws-sdk/util-retry package to wrap your API calls with retry logic. This ensures that if you hit a 429, the script will wait and try again instead of failing immediately.

Add a Cool-Down Period in Your Pipeline #

As a quick workaround, add a delay between the Serverless deploy and your script in your Bitbucket pipeline. This gives AWS time to process the deployment requests and reset some of the rate limit counters.

Add a step like this:

- step:
    name: Deploy and Update API Gateway
    script:
      - ./node_modules/.bin/serverless deploy -s devme
      - sleep 30 # Wait 30 seconds before running the script
      - node your-update-script.js

Note that this is a temporary fix—retry logic is better for consistency, but this can help reduce immediate hits.

Adjust API Gateway Throttling Settings (If Applicable) #

If the 429 is coming from the API Gateway's runtime throttling (not the management API rate limits), you can adjust the throttling settings for your API stage in the AWS Console:

• Go to API Gateway > Your API > Stages > Select your devme stage
• Under "Throttling", increase the "Rate limit" (requests per second) and "Burst limit" values
• Save the changes

Keep in mind this only affects runtime API requests, not the management API calls your script uses to create models.

Limit Concurrent Pipeline Runs #

If multiple pipelines are triggering this issue, configure Bitbucket to limit the number of concurrent runs for this workflow. You can do this in your pipeline settings:

• Go to your Bitbucket repo > Settings > Pipelines > Settings
• Under "Concurrency", set a maximum number of concurrent pipeline runs (e.g., 1 or 2)
• This prevents multiple instances from flooding AWS with requests at the same time

Request a Rate Limit Increase from AWS #

If you've tried the above and still hit 429s, you can request a higher rate limit for the API Gateway management operations via the AWS Support Console. Provide details about your use case, how many requests you need to send, and why the current limit isn't sufficient. AWS typically approves reasonable requests for legitimate use cases.

内容的提问来源于stack exchange，提问作者Steven