Node.js-Express队列处理第三方API请求并发限制问题

Hey there, looks like you're hitting a classic concurrency cap issue with that third-party API. When 6+ users hit your endpoint at the same time, the extra requests get rejected by the third party—you'll probably see 429 Too Many Requests errors, or failed calls that break your API's ability to return valid results to users.

Here are a couple of solid solutions to handle this, depending on your preference:

1. Use a Concurrency Limiting Library (Quick & Clean) #

The easiest way to tame this is with a dedicated library like p-limit—it handles all the queueing and concurrency control out of the box.

Steps to implement: #

• First, install the package:

  npm install p-limit
  

• Initialize the limiter with the third-party's allowed concurrency (5 in your case):

  const pLimit = require('p-limit');
  // Cap concurrent requests to 5, matching the third-party's limit
  const limit = pLimit(5);
  
  // Wrap your third-party API call in the limiter
  const limitedBalanceCheck = limit((marketCode) => {
    return externalAPI.balance(marketCode);
  });
  

• Update your route handler to use the wrapped function:

  if (acc_balance > req.body.amount) {
    try {
      const balanceResult = await limitedBalanceCheck(req.body.currency.market_code);
      return balanceResult; // Or send this back in your API response
    } catch (err) {
      // Handle errors from the third party (timeouts, rejections, etc.)
      throw new Error(`Failed to fetch balance from third-party API: ${err.message}`);
    }
  } else {
    throw new Error("Insufficient Funds");
  }
  

This way, no matter how many concurrent users you have, p-limit will keep active requests to the third party at 5 or less. Extra requests will queue up and execute as soon as a slot frees up.

2. Roll Your Own Simple Request Queue (No Extra Dependencies) #

If you'd rather avoid adding a new package, you can build a basic queue system to manage concurrency manually:

// Track active requests and the queue of pending ones
const requestQueue = [];
let activeRequests = 0;
const MAX_ALLOWED = 5;

function processNextRequest() {
  // Only process if we have room and pending requests
  while (requestQueue.length > 0 && activeRequests < MAX_ALLOWED) {
    const { marketCode, resolve, reject } = requestQueue.shift();
    activeRequests++;

    externalAPI.balance(marketCode)
      .then(result => resolve(result))
      .catch(err => reject(err))
      .finally(() => {
        activeRequests--;
        // After finishing one request, check the queue again
        processNextRequest();
      });
  }
}

// Wrapper function to add requests to the queue
function limitedBalanceCheck(marketCode) {
  return new Promise((resolve, reject) => {
    requestQueue.push({ marketCode, resolve, reject });
    processNextRequest();
  });
}

Then use limitedBalanceCheck in your route handler just like the library example. This is a bare-bones implementation, but it gets the job done for basic use cases.

Bonus Tips #

• Add timeout handling to your third-party API calls—if a request hangs, it'll block your queue. Use something like Promise.race with a timeout promise to avoid this.
• Log queue activity (e.g., number of pending requests, when requests start/finish) to debug bottlenecks.
• If possible, reach out to the third-party API provider to see if you can get a higher concurrency limit—this is the most straightforward fix if it's an option (though it may require a paid plan or business agreement).

内容的提问来源于stack exchange，提问作者Joshua Majebi