Kubernetes集群内Spring Boot无法连接Cassandra求助

同一default命名空间下，Cassandra Pod运行正常，但Spring Boot应用连接时抛出连接拒绝异常。

异常信息 #

Suppressed:
io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: cassandra/10.111.117.185:32532
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:710)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.netty.channel.StacklessClosedChannelException: null

Cassandra集群K8s配置 #

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  type: NodePort
  ports:
  - port: 9042
    targetPort: 9042
    protocol: TCP
    nodePort: 32532
  selector:
    app: cassandra
---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  serviceName: cassandra
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      terminationGracePeriodSeconds: 1800
      containers:
      - name: cassandra
        image: cassandra:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 7000
          name: intra-node
        - containerPort: 7001
          name: tls-intra-node
        - containerPort: 7199
          name: jmx
        - containerPort: 9042
          name: cql
        resources:
          limits:
            cpu: "500m"
            memory: 1Gi
          requests:
            cpu: "500m"
            memory: 1Gi
        securityContext:
          capabilities:
            add:
              - IPC_LOCK
        lifecycle:
          preStop:
            exec:
              command: 
              - /bin/sh
              - -c
              - nodetool drain
        env:
          - name: MAX_HEAP_SIZE
            value: 512M
          - name: HEAP_NEWSIZE
            value: 100M
          - name: CASSANDRA_SEEDS
            value: "cassandra-0.cassandra.default.svc.cluster.local"
          - name: CASSANDRA_CLUSTER_NAME
            value: "K8SCassandra"
          - name: CASSANDRA_DC
            value: "DC1-K8SCassandra"
          - name: CASSANDRA_RACK
            value: "Rack1-K8SCassandra"
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
        
        volumeMounts:
        - name: cassandra-data
          mountPath: /cassandra_data
  volumeClaimTemplates:
  - metadata:
      name: cassandra-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: standard
      resources:
        requests:
          storage: 1Gi

Spring Boot应用K8s配置 #

apiVersion: v1
kind: Service
metadata:
  name: service-cassandraapp
  labels:
     app: cassandraapp
spec:
  selector:
    app: cassandraapp
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      nodePort: 32588
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-cassandraapp
  labels:
    app: cassandraapp
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  selector:
    matchLabels:
      app: cassandraapp
  template:
    metadata:
      labels:
        app: cassandraapp
    spec:
      containers:
      - name: cassandraapp
        image: ek/cassandraapp:latest
        ports:
          - containerPort: 8080
        resources:
          limits:
            memory: "1Gi"
            cpu: "1000m"
          requests:
            memory: "256Mi"
            cpu: "500m"
        env:
          - name: CONFIG_CASSANDRA_HOST
            value: "cassandra"
          - name: CONFIG_CASSANDRA_PORT
            value: "32532"  

Spring Boot应用application.properties #

spring.data.cassandra.local-datacenter=datacenter1
spring.data.cassandra.keyspace-name=testkeyspace
spring.data.cassandra.port=${CONFIG_CASSANDRA_PORT}
spring.data.cassandra.contact-points=${CONFIG_CASSANDRA_HOST}
spring.data.cassandra.username=cassandra
spring.data.cassandra.password=cassandra
spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS

问题排查与解决 #

1. 修正端口配置错误 #

Spring Boot应用当前配置的32532是Cassandra Service的NodePort，仅用于集群外部访问。同一K8s命名空间内的应用需使用Service的ClusterIP端口（即Cassandra Service中定义的port: 9042）。

修改Spring Boot Deployment的环境变量：

env:
  - name: CONFIG_CASSANDRA_HOST
    value: "cassandra"
  - name: CONFIG_CASSANDRA_PORT
    value: "9042"  # 替换为ClusterIP端口

2. 统一数据中心名称 #

Cassandra配置中设置的CASSANDRA_DC=DC1-K8SCassandra，但Spring Boot的spring.data.cassandra.local-datacenter=datacenter1，两者不一致会导致连接失败。

修改Spring Boot的application.properties：

spring.data.cassandra.local-datacenter=DC1-K8SCassandra

3. 验证Cassandra服务可用性 #

在Spring Boot Pod内执行以下命令测试连接：

# 使用telnet测试
telnet cassandra 9042
# 或使用nc
nc -zv cassandra 9042

若连通失败，检查Cassandra Service的selector是否匹配Pod标签，以及Cassandra Pod是否正常监听9042端口：

# 进入Cassandra Pod
kubectl exec -it cassandra-0 -- /bin/bash
# 检查端口监听状态
netstat -tulpn | grep 9042

4. 确认Cassandra集群就绪 #

Cassandra StatefulSet启动后需时间完成集群初始化，执行以下命令检查集群状态：

kubectl exec -it cassandra-0 -- nodetool status

确保所有节点状态为UN（Up/Normal）后，再重启Spring Boot应用。

内容的提问来源于stack exchange，提问作者efk