GKE部署带Socket.io的Node.js服务遇30秒自动重启问题求助

问题描述 #

在GKE部署集成Socket.io的Node.js服务，最初采用NodePort+外部HTTPS Ingress架构时，Socket连接每隔30秒自动断开重启。已设置timeoutSec: 3600和sessionAffinity: ClientIP但无效；切换为External LoadBalancer服务后，断开问题解决，但无法添加SSL证书；重新配置Ingress则Socket重启问题重现。需解决两个问题：无需Ingress添加SSL的方法，或彻底修复Ingress下Socket连接重启问题。

现有配置文件 #

deployment.yaml #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: lyxa-backend-test
  namespace: default
  labels:
    app: lyxa-backend-test
spec:
  selector:
    matchLabels:
      app: lyxa-backend-test
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
  replicas: 2 # number of pods
  template:
    metadata:
      labels:
        app: lyxa-backend-test
    spec:
      containers:
        - name: lyxa-backend-test
          image: gcr.io/for-poc-325210/lyxa-backend-test:v1
          imagePullPolicy: Always
          ports:
            - containerPort: 5001
              protocol: TCP
          env:
            - name: TZ
              value: Europe/Stockholm
      imagePullSecrets:
        - name: dropimagepullkey

service.yaml #

apiVersion: v1
kind: Service
metadata:
  name: lyxa-backend-test-service
  annotations:
    cloud.google.com/app-protocols: |
      {
        "https": "HTTPS",
        "http": "HTTP"
      }
    cloud.google.com/neg: '{"ingress": true}'
spec:
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 5001
    - protocol: TCP
      port: 443
      targetPort: 5001
  selector:
    app: lyxa-backend-test

backendConfig.yaml #

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: lyxa-backend-test-backendconfig
spec:
  timeoutSec: 2147483647
  sessionAffinity:
    affinityType: "CLIENT_IP"
  connectionDraining:
    drainingTimeoutSec: 3600

ingress.yaml #

apiVersion: "networking.k8s.io/v1"
kind: "Ingress"
metadata:
  name: "lyxa-backend-test-ingress"
  namespace: "default"
  annotations:
    networking.gke.io/managed-certificates: "lyxa-backend-test-ssl-certificate"
    kubernetes.io/ingress.class: "gce-internal"
    cloud.google.com/backend-config: '{"default": "lyxa-backend-test-backendconfig"}'
spec:
  defaultBackend:
    service:
      name: "lyxa-backend-test-service"
      port:
        number: 80
status:
  loadBalancer: {}

解决方案 #

一、彻底修复Ingress下Socket.io连接重启问题 #

1. 修正BackendConfig超时与关联配置

GCE HTTP(S)负载均衡最大超时为3600秒，你设置的2147483647会被自动截断，需明确设置为3600秒。同时必须将BackendConfig关联到Service（当前仅关联到Ingress，这是核心问题之一）：

• 更新backendConfig.yaml的timeoutSec：

  spec:
    timeoutSec: 3600
  

• 更新service.yaml的annotations，添加BackendConfig关联：

  metadata:
    annotations:
      cloud.google.com/backend-config: '{"default": "lyxa-backend-test-backendconfig"}'
  

2. 启用WebSocket支持

在Ingress中添加WebSocket启用注解：

• 更新ingress.yaml的annotations：

  metadata:
    annotations:
      networking.gke.io/websocket-services: "lyxa-backend-test-service"
  

3. 完善会话亲和性配置

除BackendConfig外，需在Service层面设置会话亲和性，确保请求始终路由到同一Pod：

• 更新service.yaml的spec：

  spec:
    sessionAffinity: ClientIP
    sessionAffinityConfig:
      clientIP:
        timeoutSeconds: 3600
  

4. 优化Socket.io服务端配置

在Node.js代码中优先使用WebSocket传输，调整心跳参数避免超时：

const io = require('socket.io')(server, {
  transports: ['websocket', 'polling'], // 优先WebSocket
  pingTimeout: 60000, // 心跳超时设为60秒
  pingInterval: 25000 // 心跳间隔设为25秒
});

二、无需Ingress添加SSL证书的方法 #

1. 给LoadBalancer服务关联Managed Certificate

通过BackendConfig将GCP托管证书直接绑定到LoadBalancer服务：

• 确保已创建托管证书lyxa-backend-test-ssl-certificate
• 更新service.yaml的annotations，添加证书关联：

  metadata:
    annotations:
      networking.gke.io/managed-certificates: "lyxa-backend-test-ssl-certificate"
  

• 更新backendConfig.yaml，添加SSL证书配置（先通过gcloud compute ssl-certificates describe lyxa-backend-test-ssl-certificate --format="value(id)"获取证书ID）：

  spec:
    ssl:
      sslCertificates:
      - id: "<你的托管证书ID>"
  

2. Pod内部直接处理HTTPS

将SSL证书挂载到Pod，让Node.js服务自行处理HTTPS：

• 创建TLS Secret存储证书：

  kubectl create secret tls lyxa-backend-tls --cert=fullchain.pem --key=privkey.pem
  

• 更新deployment.yaml，挂载Secret并添加环境变量：

  spec:
    containers:
      - name: lyxa-backend-test
        ports:
          - containerPort: 5001
          - containerPort: 443 # 添加HTTPS端口
        volumeMounts:
          - name: tls-secret
            mountPath: /etc/tls
            readOnly: true
        env:
          - name: SSL_CERT_PATH
            value: /etc/tls/tls.crt
          - name: SSL_KEY_PATH
            value: /etc/tls/tls.key
    volumes:
      - name: tls-secret
        secret:
          secretName: lyxa-backend-tls
  

• 修改Node.js代码启用HTTPS：

  const fs = require('fs');
  const https = require('https');
  const socketIo = require('socket.io');
  
  const sslOptions = {
    key: fs.readFileSync(process.env.SSL_KEY_PATH),
    cert: fs.readFileSync(process.env.SSL_CERT_PATH)
  };
  
  const httpsServer = https.createServer(sslOptions);
  const io = socketIo(httpsServer);
  
  httpsServer.listen(443, () => {
    console.log('HTTPS server running on port 443');
  });
  

内容的提问来源于stack exchange，提问作者Sajib Uzzaman