分布式原子列表：微服务间同步——Back服务多副本扩容资源冲突咨询

Hey Jordi, this is a classic challenge when scaling worker-style services—glad you’re thinking ahead about scalability! Let’s walk through the most reliable solutions to stop those duplicate processing issues:

1. Use a Distributed Lock #

When your Back instance requests a resource ID, first attempt to acquire a distributed lock tied to that ID (tools like Redis with SETNX or Redlock work great here). Only the instance that successfully grabs the lock gets to process the resource.

• Make sure to set a reasonable timeout on the lock (longer than your typical processing time) to avoid deadlocks if an instance crashes mid-processing.
• After processing completes, explicitly release the lock so the next resource can be picked up.

2. Shift to a Task Queue Middleware #

This is the most scalable and widely adopted approach. Instead of Back instances polling Front for resource IDs, have Front push all pending resource IDs into a dedicated task queue (like RabbitMQ, Kafka, or Redis Queue).

• Each Back instance pulls tasks from the queue, and the queue ensures each task is only delivered to one consumer at a time.
• Use the queue’s acknowledgment mechanism: the Back instance sends an ACK only after processing finishes. If it crashes without acknowledging, the queue will re-deliver the task (you can set retry limits to avoid infinite loops).

3. Centralized Task Assignment via Front #

Modify your workflow so Front acts as a task coordinator instead of a passive responder:

• Front maintains a list of available Back instances and tracks which resource IDs have been assigned.
• When a Back instance requests work, Front assigns an unclaimed resource ID and marks it as "assigned" in its state.
• This reduces polling overhead but adds more logic to Front, increasing coupling between services.

4. Atomic Database State Marking #

If you’re using a database to track resource statuses, use atomic operations to claim resources:

• Add a status field to your resource table (e.g., pending, processing, completed).
• When a Back instance wants to process a resource, run an atomic UPDATE query like:

  UPDATE resources SET status = 'processing' WHERE id = ? AND status = 'pending'
  

• Only the instance that successfully updates the status gets to process the resource. Set a timeout to reset processing statuses if an instance fails to complete the task.

Recommendation #

For long-term scalability, go with the task queue middleware approach—it decouples your services, handles retries gracefully, and makes scaling Back instances trivial (just spin up more consumers).

内容的提问来源于stack exchange，提问作者Jordi