寻求兼具RethinkDB低延迟Changefeeds与高吞吐量的开源数据库方案

Awesome question—this is such a common pain point when you need both blistering write throughput and real-time change notifications. Let’s walk through the best open-source options (both single tools and stack combinations) that check both boxes for you:

1. Cassandra + Debezium + Kafka (Double down on Cassandra’s throughput with real-time CDC) #

If you already love Cassandra’s write performance, you don’t have to give that up to get changefeeds. Here’s how to make it work:

• Enable CDC (Change Data Capture) on your Cassandra tables—this lets the database track all writes/updates/deletes in its commit log.
• Use Debezium, an open-source CDC tool, to tail Cassandra’s commit log, turn those raw entries into structured change events (like "user X was updated"), and push them to Kafka.
• Subscribers can then consume those Kafka topics directly. Kafka is built to handle thousands of concurrent subscribers with low latency, and you can even add filtering or transformation logic (like only sending updates to specific users) via Kafka Streams if you need it.

The latency here is typically sub-100ms, which matches RethinkDB’s changefeeds, and you keep Cassandra’s 1M+ writes/sec throughput. Just note that you’ll need to tweak Cassandra’s CDC settings to balance performance and retention.

2. MongoDB Sharded Cluster + Change Streams (All-in-one simplicity with serious scaling) #

MongoDB’s Change Streams are basically RethinkDB changefeeds on steroids, and a sharded MongoDB cluster can easily hit 500k+ writes/sec (and more with proper sharding)—way better than RethinkDB’s limits.

• Set up a sharded MongoDB cluster (split your data across multiple nodes to scale writes).
• Enable Change Streams on your collections, databases, or even the entire cluster. Subscribers can open a persistent connection to get real-time updates as soon as data changes.
• You can filter changes by document ID, operation type (insert/update/delete), or even run aggregations directly on the stream to cut down on the data sent to subscribers.

This is the simplest option if you don’t want to mess with multiple tools. It’s not quite as fast as Cassandra for writes, but it’s more than enough for most use cases, and Change Streams handle thousands of concurrent subscribers without breaking a sweat.

3. Kafka + Kafka Streams + RocksDB (Build your own custom PubSub backend) #

If you want full control over every part of your system, build a custom backend around Kafka:

• Use Kafka as your write layer—it handles 1M+ writes/sec effortlessly, even on commodity hardware.
• Use Kafka Streams to process and index that data into materialized views (using RocksDB as the embedded state store) if you need fast querying.
• Expose changefeeds by having subscribers consume either the raw Kafka topics (for all writes) or the processed Streams topics (for filtered/aggregated changes).

This requires more engineering work, but it’s incredibly flexible. You can tailor it exactly to your PubSub needs, and scale writes and subscribers independently. Plus, Kafka’s consumer model is built to handle massive numbers of subscribers with low latency.

4. NATS JetStream (Lightweight, high-performance PubSub with persistence) #

If your primary focus is PubSub (and you don’t need complex database querying), NATS JetStream is a fantastic option. It’s built for speed:

• It supports millions of writes/sec, sub-millisecond latency for message delivery, and tens of thousands of concurrent subscribers.
• You can write directly to JetStream streams, and subscribers can subscribe with filters to get only the changes they care about. It also supports durable subscriptions, so subscribers don’t miss any changes if they disconnect.

The only catch is that if you need to run complex queries on your data, you’ll need to add a separate database for indexing. But if you just need high-throughput writes + real-time change propagation, JetStream is super lightweight and easy to set up.

Final Call #

Which option you choose depends on your existing stack and priorities:

• If you’re already using Cassandra: Go with Debezium + Kafka to add changefeeds without losing throughput.
• If you want simplicity: MongoDB Change Streams are a no-brainer—all-in-one, scalable, and familiar.
• If you need full control: Build your own system with Kafka Streams.
• If PubSub is your main focus: NATS JetStream is hard to beat for speed and simplicity.

内容的提问来源于stack exchange，提问作者rygo6