Kubernetes下MongoDB副本集MongoNotPrimaryException问题求助

环境与问题 #

• Kubernetes部署MongoDB副本集：2个数据Pod + 1个仲裁节点
• SpringBoot应用通过Reactive Mongo Repository交互，连接URL：

  mongodb://{{mongoURL}}/?directConnection=true&serverSelectionTimeoutMS=2000&readPreference=primary
  

• 持续收到错误：

  Command failed with error 10107 (NotWritablePrimary): 'not primary'.

已执行排查 #

• 确认Kubernetes中MongoDB副本集及仲裁节点配置无误
• 检查SpringBoot应用Reactive Mongo Repository代码逻辑

实际结果 #

连接串已指定readPreference=primary，但写入操作仍持续触发MongoNotPrimaryException，不符合预期。

MongoDB Pod日志 #

{"t":{"$date":"2023-11-09T09:05:48.481+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:54236","uuid":{"uuid":{"$uuid":"87592666-c1a1-4d21-9f2d-9e992d8ad268"}},"connectionId":40918,"connectionCount":32}}
{"t":{"$date":"2023-11-09T09:05:48.481+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:54244","uuid":{"uuid":{"$uuid":"795de4a0-3d8b-4688-a046-e6e64330752f"}},"connectionId":40919,"connectionCount":33}}
{"t":{"$date":"2023-11-09T09:05:48.482+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn40918","msg":"client metadata","attr":{"remote":"127.0.0.1:54236","client":"conn40918","doc":{"application":{"name":"mongosh 1.10.6"},"driver":{"name":"nodejs|mongosh","version":"5.7.0|1.10.6"},"platform":"Node.js v16.20.2, LE","os":{"name":"linux","architecture":"x64","version":"5.15.0-1041-azure","type":"Linux"}}}}
{"t":{"$date":"2023-11-09T09:05:48.482+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn40919","msg":"client metadata","attr":{"remote":"127.0.0.1:54244","client":"conn40919","doc":{"application":{"name":"mongosh 1.10.6"},"driver":{"name":"nodejs|mongosh","version":"5.7.0|1.10.6"},"platform":"Node.js v16.20.2, LE","os":{"name":"linux","architecture":"x64","version":"5.15.0-1041-azure","type":"Linux"}}}}
{"t":{"$date":"2023-11-09T09:05:48.485+00:00"},"s":"I",  "c":"NETWORK",  "id":6788700, "ctx":"conn40918","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":3}}
{"t":{"$date":"2023-11-09T09:05:48.485+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:54252","uuid":{"uuid":{"$uuid":"f1419771-32a0-4760-9e68-6736d27f7a71"}},"connectionId":40920,"connectionCount":34}}
{"t":{"$date":"2023-11-09T09:05:48.489+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn40920","msg":"client metadata","attr":{"remote":"127.0.0.1:54252","client":"conn40920","doc":{"application":{"name":"mongosh 1.10.6"},"driver":{"name":"nodejs|mongosh","version":"5.7.0|1.10.6"},"platform":"Node.js v16.20.2, LE","os":{"name":"linux","architecture":"x64","version":"5.15.0-1041-azure","type":"Linux"}}}}
{"t":{"$date":"2023-11-09T09:05:48.499+00:00"},"s":"I",  "c":"NETWORK",  "id":6788700, "ctx":"conn40920","msg":"Received first command on ingress connection since session start or auth handshake","attr":{"elapsedMillis":10}}
{"t":{"$date":"2023-11-09T09:05:49.503+00:00"},"s":"I",  "c":"-",        "id":20883,   "ctx":"conn40917","msg":"Interrupted operation as its client disconnected","attr":{"opId":1375581}}
{"t":{"$date":"2023-11-09T09:05:49.503+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn40920","msg":"Connection ended","attr":{"remote":"127.0.0.1:54252","uuid":{"uuid":{"$uuid":"f1419771-32a0-4760-9e68-6736d27f7a71"}},"connectionId":40920,"connectionCount":33}}
{"t":{"$date":"2023-11-09T09:05:49.503+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn40918","msg":"Connection ended","attr":{"remote":"127.0.0.1:54236","uuid":{"uuid":{"$uuid":"87592666-c1a1-4d21-9f2d-9e992d8ad268"}},"connectionId":40918,"connectionCount":32}}
{"t":{"$date":"2023-11-09T09:05:49.503+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn40919","msg":"Connection ended","attr":{"remote":"127.0.0.1:54244","uuid":{"uuid":{"$uuid":"795de4a0-3d8b-4688-a046-e6e64330752f"}},"connectionId":40919,"connectionCount":31}}
{"t":{"$date":"2023-11-09T09:05:49.503+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn40917","msg":"Connection ended","attr":{"remote":"127.0.0.1:54232","uuid":{"uuid":{"$uuid":"a571fb30-30ec-4a87-83e8-0b0ed0817385"}},"connectionId":40917,"connectionCount":30}}

问题分析与解决建议 #

核心分析 #

1. directConnection=true的副作用：该参数强制客户端直接连接到指定的{{mongoURL}}节点，而非通过副本集发现机制获取主节点地址。如果{{mongoURL}}指向从节点/仲裁节点，或主节点切换后客户端未感知，就会触发写入错误。
2. 副本集发现机制未生效：连接串未指定replicaSet参数，Java驱动无法自动发现副本集的主节点，即使设置readPreference=primary也无法动态切换到当前主节点。
3. 日志无有效错误线索：提供的日志仅显示mongosh连接的网络事件，未包含应用连接的元数据或写入错误记录，无法直接定位应用连接的节点角色。

解决步骤 #

• 修改连接串，启用副本集发现：
  移除directConnection=true，添加replicaSet=<副本集名称>参数，示例：

  mongodb://{{mongoURL}}/?serverSelectionTimeoutMS=2000&readPreference=primary&replicaSet=rs0
  

  注：{{mongoURL}}应包含副本集所有节点地址（如mongo-0:27017,mongo-1:27017,mongo-arbiter:27017），驱动会自动发现主节点。
• 验证副本集主节点状态：
  进入MongoDB Pod执行命令，确认当前主节点：

  mongosh --eval "rs.status()"
  

  检查members数组中stateStr为PRIMARY的节点，确认{{mongoURL}}包含该节点地址。
• 添加写关注点保障：
  写入操作添加w=majority参数，确保写入被多数节点确认，同时避免因主节点切换导致的写入失败：

  mongodb://{{mongoURL}}/?serverSelectionTimeoutMS=2000&readPreference=primary&replicaSet=rs0&w=majority
  

• 排查应用连接节点：
  在SpringBoot应用中添加日志，输出实际连接的MongoDB节点地址，确认是否连接到主节点。
• 检查副本集网络连通性：
  在SpringBoot Pod中执行telnet或nc命令，测试到所有MongoDB节点的网络连通性，确保应用能访问主节点。

内容的提问来源于stack exchange，提问作者drprolegend