面试题：如何设计可处理百万级请求的Rest API？

Hey there! Let's break down your interview questions one by one—this is such a critical topic for building scalable APIs, so it’s awesome you’re diving into these details.

一、是否需要特殊方案？绝对需要 #

A regular single-instance REST API can’t handle millions of concurrent requests—you’ll hit bottlenecks in CPU, memory, database, or network way before reaching that scale. So yes, you need a holistic, scalable architecture rather than just tweaking a few settings.

二、核心设计要点处理百万级请求 #

Here’s what you need to focus on:

• Stateless API Design
  Make every request self-contained—don’t store session data in server memory. Use tokens like JWT or distributed caches (Redis) for session management. This lets you scale horizontally easily, since load balancers can route requests to any instance without worrying about session sync.

• Horizontal Scaling + Load Balancing
  Ditch vertical scaling (throwing more CPU/RAM at one server) once you hit its limit. Deploy multiple API instances behind a load balancer (like Nginx, HAProxy) that distributes traffic evenly. This way, you can add/remove instances based on traffic spikes.

• Smart Caching
  Reduce backend pressure with caching:

  • Use Redis/Memcached for hot data (e.g., product details, user profiles) with appropriate TTLs.
  • Leverage HTTP caching headers (Cache-Control, ETag) to let clients cache static resources or repeat responses.

• Asynchronous Processing for Non-Core Logic
  Don’t block requests on non-essential tasks. For example, send welcome emails, log analytics, or generate reports asynchronously using message queues (RabbitMQ, Kafka). This lets you return responses to users instantly and process background tasks later.

• Database Optimization
  Databases are often the biggest bottleneck:

  • Implement read-write separation (master for writes, replicas for reads).
  • Shard databases/tables by user ID, time, or other logical keys to split data across multiple servers.
  • Add proper indexes to speed up queries, and avoid long-running transactions that cause lock contention.

• API Gateway
  Use an API gateway (Spring Cloud Gateway, Kong) as a single entry point. It handles routing, rate limiting, authentication, logging, and circuit breaking—freeing your backend services to focus on business logic.

• Rate Limiting & Circuit Breaking

  • Rate Limiting: Use token bucket/leaky bucket algorithms to control incoming traffic (e.g., limit 100 requests per user per minute). This prevents sudden traffic spikes from overwhelming your system.
  • Circuit Breaking: Tools like Resilience4j or Hystrix stop sending requests to failing services temporarily. If a downstream service is down, the circuit "opens" to avoid request stacking, and retries after a cool-off period.

• Monitoring & Observability
  Track QPS, response times, error rates in real-time with tools like Prometheus + Grafana. Centralize logs with ELK Stack (Elasticsearch, Logstash, Kibana) to quickly debug issues when they pop up.

三、关于“出现问题时创建线程”的误区与正确方案 #

Your initial thought of creating threads when issues arise is actually an anti-pattern—here’s why:

Thread creation/destruction has significant overhead. Uncontrolled thread spawning leads to excessive CPU context switching, memory exhaustion, and eventually system crashes (OOM errors). It’ll make your problems worse, not better.

Correct Approaches: #

• Use Thread Pools
  Pre-configure a thread pool with a fixed or dynamically adjusted number of threads (based on CPU cores). Reuse threads to handle requests instead of creating new ones every time. For example, in Java, use ThreadPoolExecutor; Spring Boot uses a built-in thread pool for web requests by default. Thread pools let you control concurrency and avoid resource leaks.

• Adopt Asynchronous Non-Blocking IO
  Instead of one thread per request, use non-blocking IO frameworks (Spring WebFlux, Netty) or event-driven runtimes (Node.js). A single thread can handle thousands of concurrent requests by waiting for IO operations (like database calls) without blocking. This drastically improves throughput.

• Circuit Breaking (Again!)
  When a dependency fails, use circuit breakers to stop sending requests and return fallback responses immediately. This prevents thread pools from getting clogged with pending requests to failing services.

内容的提问来源于stack exchange，提问作者Indiver kumar