Hey there! Let's break down why your throughput isn't scaling linearly when adding a second queue and how to fix it. Based on your setup (RabbitMQ 3.7.3 + Java client 5.1.2), here are the key areas to optimize:

1. Fix Producer-Side Bottlenecks #

The most common culprit here is serialized message publishing if you're using a single thread/Channel to write to both queues. Here's how to fix it:

• Use Separate Channels per Queue: Channels are thread-unsafe, so sending to two queues from one Channel forces operations to run sequentially. Create a dedicated Channel for each queue, and pair each with an independent thread pool to publish messages in parallel. This lets each queue's traffic flow without blocking the other.
• Enable Asynchronous Publisher Confirms: Instead of waiting for a synchronous confirm after every message, call channel.confirmSelect() and register a ConfirmListener to handle acks/nacks in the background. This eliminates blocking and lets your producer keep sending messages without waiting for server responses.
• Batch Message Publishing: Cut down on TCP overhead by batching messages (e.g., 100-500 messages per batch) before calling basicPublish. You can manually accumulate messages in a buffer and flush it when it hits a size threshold or time limit—just balance batch size to avoid excessive latency.

2. Tune RabbitMQ Server Configuration #

Your server might be hitting resource limits that cap total throughput:

• Memory & Disk Limits: Check if RabbitMQ is triggering flow control due to memory/disk constraints. Update your rabbitmq.conf (or rabbitmq-env.conf for older setups):
  • Set vm_memory_high_watermark.relative = 0.7 (adjust based on your server's total RAM) to avoid premature flow control.
  • Ensure disk_free_limit.relative = 1.0 (or set an absolute value) so the server doesn't throttle producers when disk space is low.
• Erlang VM Optimizations: Tweak Erlang's runtime to handle more concurrent operations:
  • Add +P 1048576 to increase the max number of Erlang processes, supporting more connections and channels.
  • Enable async IO with +K true and set +A 64 (async thread pool size) to improve disk and network handling.
• Network & IO Threads: Increase TCP acceptors and IO threads to handle more concurrent connections:
  • Set num_acceptors.tcp = 8 (match to 2x your CPU core count)
  • Raise num_io_threads = 128 (default is 64; adjust if you have high IO load)

3. Optimize Queue Settings #

Queue-specific configurations can also limit scalability:

• Disable Unnecessary Persistence: If your use case doesn't require message durability, set durable=false when declaring queues. Persistent messages force disk writes, which are a major throughput bottleneck—disabling this can drastically boost per-queue performance.
• Use Lazy Queues (If Persistence Is Required): If you need to keep persistence, enable lazy queues by adding x-queue-mode=lazy during queue declaration. This forces messages to be written to disk immediately instead of holding them in memory, reducing memory pressure and avoiding flow control that throttles producers.
• Adjust Queue Index Embedding: Tweak x-queue-index-embed-msgs-below (default 4096 bytes) to embed small messages directly into the queue index. If your messages are smaller than the default, increase this value to reduce separate disk writes for tiny messages.

4. Validate with Monitoring & Testing #

• Monitor RabbitMQ Metrics: Use the RabbitMQ management UI or rabbitmqctl to check:
  • Queue message ingress rates to confirm each queue is receiving traffic as expected.
  • Server CPU, memory, and disk IO usage—if disk IO is pegged, that's your bottleneck.
• Isolate Tests: First test if a single queue can exceed 25k msg/s with a multi-threaded producer. If yes, your original single-queue test was limited by producer concurrency, not RabbitMQ. Then test two queues with independent producer threads to verify linear scaling.

By addressing these areas, you should be able to get each queue to hit 25k msg/s, achieving the linear throughput you're looking for.

内容的提问来源于stack exchange，提问作者Carbonrock