如何在一个消费者组中创建Kafka消费者以读取多个主题？

步骤1：创建不同分区数的主题 #

我创建了两个分区数不同的主题，执行命令如下：

kafka-topics.sh --bootstrap-server localhost:9092 --topic first_topic --create --partitions 5 --replication-factor 1

以及

kafka-topics.sh --bootstrap-server localhost:9092 --topic second_topic --create --partitions 4 --replication-factor 1

步骤2 - 尝试1：使用竖线分隔符的白名单监听主题 #

创建三个消费者监听上述两个主题，在三个不同终端分别执行以下命令：

kafka-console-consumer.sh --bootstrap-server localhost:9092  --whitelist 'first-topic|second_topic' --property print.offset=true --property print.partition=true --group my-first-application

查看消费者组信息：

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-first-application

GROUP                TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                           HOST            CLIENT-ID
my-first-application second_topic    0          0               0               0               console-consumer-3db678b0-931c-4e22-9632-040df2d64d62 /127.0.0.1      console-consumer
my-first-application second_topic    1          0               0               0               console-consumer-3db678b0-931c-4e22-9632-040df2d64d62 /127.0.0.1      console-consumer
my-first-application second_topic    3          0               0               0               console-consumer-ae8a6f87-9ac7-4eb5-a9eb-0445254e2d24 /127.0.0.1      console-consumer
my-first-application second_topic    2          0               0               0               console-consumer-58b4d231-58cf-4c63-a3f1-fbe4d50873e3 /127.0.0.1      console-consumer
my-first-application first_topic     2          0               0               0               -                                                     -               -
my-first-application first_topic     1          0               0               0               -                                                     -               -
my-first-application first_topic     4          0               0               0               -                                                     -               -
my-first-application first_topic     0          0               0               0               -                                                     -               -
my-first-application first_topic     3          0               0               0               -                                                     -               -

结果：该消费者组仅分配了second_topic的分区。

步骤2 - 尝试2：使用逗号分隔符的白名单监听主题 #

关闭活跃消费者后，在三个不同终端分别执行以下命令：

kafka-console-consumer.sh --bootstrap-server localhost:9092  --whitelist 'first-topic,second_topic' --property print.offset=true --property print.partition=true --group my-first-application

查看消费者组信息：

kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-first-application

GROUP                TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
my-first-application second_topic    0          0               0               0               -               -               -
my-first-application first_topic     1          0               0               0               -               -               -
my-first-application second_topic    2          0               0               0               -               -               -
my-first-application first_topic     4          0               0               0               -               -               -
my-first-application first_topic     0          0               0               0               -               -               -
my-first-application second_topic    1          0               0               0               -               -               -
my-first-application first_topic     3          0               0               0               -               -               -
my-first-application second_topic    3          0               0               0               -               -               -
my-first-application first_topic     2          0               0               0               -               -               -

内容的提问来源于stack exchange，提问作者Wrapper