Cloud Run部署Kubedoom服务失败：端口监听异常求助

尝试部署Kubedoom服务到Cloud Run，该服务在GKE和Helm环境下可正常运行，但Cloud Run部署时持续报错。

Cloud Run使用的YAML配置 #

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: kubedoom-kubedoom-chart
  namespace: 'projectnumber'
spec:
  template:
    metadata:
      labels:
        app: kubedoom-chart
    spec:
      containers:
        #      - env:
        # - name: NAMESPACE
        #  value: nginx
        image: gcr.io/alealferez-project-1/kubedoom:v5
        name: kubedoom-chart
          # ports:
          # - containerPort: 5900
          #name: http1
          #protocol: TCP
          #hostPort: 5900
          #listenAddress: "127.0.0.1"

GKE上正常运行的YAML配置 #

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "2"
    meta.helm.sh/release-name: kubedoom
    meta.helm.sh/release-namespace: kubedoom
  creationTimestamp: "2023-02-28T17:56:47Z"
  generation: 2
  labels:
    app.kubernetes.io/instance: kubedoom
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubedoom-chart
    app.kubernetes.io/version: 1.16.0
    helm.sh/chart: kubedoom-chart-0.1.0
  managedFields:
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:meta.helm.sh/release-name: {}
          f:meta.helm.sh/release-namespace: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/instance: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/name: {}
          f:app.kubernetes.io/version: {}
          f:helm.sh/chart: {}
      f:spec:
        f:progressDeadlineSeconds: {}
        f:replicas: {}
        f:revisionHistoryLimit: {}
        f:selector: {}
        f:strategy:
          f:rollingUpdate:
            .: {}
            f:maxSurge: {}
            f:maxUnavailable: {}
          f:type: {}
        f:template:
          f:metadata:
            f:labels:
              .: {}
              f:app.kubernetes.io/instance: {}
              f:app.kubernetes.io/name: {}
          f:spec:
            f:containers:
              k:{"name":"kubedoom-chart"}:
                .: {}
                f:image: {}
                f:imagePullPolicy: {}
                f:name: {}
                f:ports:
                  .: {}
                  k:{"containerPort":5900,"protocol":"TCP"}:
                    .: {}
                    f:containerPort: {}
                    f:name: {}
                    f:protocol: {}
                f:resources:
                  .: {}
                  f:limits:
                    .: {}
                    f:cpu: {}
                    f:memory: {}
                  f:requests:
                    .: {}
                    f:cpu: {}
                    f:memory: {}
                f:securityContext: {}
                f:terminationMessagePath: {}
                f:terminationMessagePolicy: {}
            f:dnsPolicy: {}
            f:restartPolicy: {}
            f:schedulerName: {}
            f:securityContext: {}
            f:serviceAccount: {}
            f:serviceAccountName: {}
            f:terminationGracePeriodSeconds: {}
    manager: helm
    operation: Update
    time: "2023-02-28T17:56:47Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:spec:
        f:template:
          f:spec:
            f:containers:
              k:{"name":"kubedoom-chart"}:
                f:env:
                  .: {}
                  k:{"name":"NAMESPACE"}:
                    .: {}
                    f:name: {}
                    f:value: {}
    manager: GoogleCloudConsole
    operation: Update
    time: "2023-02-28T18:05:13Z"
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:deployment.kubernetes.io/revision: {}
      f:status:
        f:availableReplicas: {}
        f:conditions:
          .: {}
          k:{"type":"Available"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
          k:{"type":"Progressing"}:
            .: {}
            f:lastTransitionTime: {}
            f:lastUpdateTime: {}
            f:message: {}
            f:reason: {}
            f:status: {}
            f:type: {}
        f:observedGeneration: {}
        f:readyReplicas: {}
        f:replicas: {}
        f:updatedReplicas: {}
    manager: kube-controller-manager
    operation: Update
    subresource: status
    time: "2023-02-28T18:06:39Z"
  name: kubedoom-kubedoom-chart
  namespace: kubedoom
  resourceVersion: "100217215"
  uid: e7e0870a-5782-4926-8aac-eb0d2105f720
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: kubedoom
      app.kubernetes.io/name: kubedoom-chart
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: kubedoom
        app.kubernetes.io/name: kubedoom-chart
    spec:
      containers:
      - env:
        - name: NAMESPACE
          value: nginx
        image: ghcr.io/storax/kubedoom:0.6.0
        imagePullPolicy: IfNotPresent
        name: kubedoom-chart
        ports:
        - containerPort: 5900
          name: vnc
          protocol: TCP
        resources:
          limits:
            cpu: 500m
            memory: 500Mi
          requests:
            cpu: 500m
            memory: 500Mi
        securityContext: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler

收到的错误信息 #

Revision 'kubedoom-kubedoom-chart-ltzkf' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable

解决方案 #

1. 端口匹配配置 #

Cloud Run默认要求容器监听PORT环境变量指定的端口（默认8080），但Kubedoom默认使用5900端口提供VNC服务。需要调整配置让Cloud Run识别容器的5900端口：

修改Cloud Run的Service YAML如下：

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: kubedoom-kubedoom-chart
  namespace: 'projectnumber'
spec:
  template:
    metadata:
      labels:
        app: kubedoom-chart
    spec:
      containers:
        - env:
            - name: NAMESPACE
              value: nginx
          image: gcr.io/alealferez-project-1/kubedoom:v5
          name: kubedoom-chart
          ports:
            - containerPort: 5900
              name: vnc
              protocol: TCP
          resources:
            limits:
              cpu: 500m
              memory: 500Mi
            requests:
              cpu: 500m
              memory: 500Mi

或者使用gcloud命令部署时直接指定端口：

gcloud run deploy kubedoom-kubedoom-chart \
  --image gcr.io/alealferez-project-1/kubedoom:v5 \
  --port 5900 \
  --set-env-vars NAMESPACE=nginx

2. 权限配置 #

Kubedoom需要访问Kubernetes API来删除指定Namespace下的资源。在Cloud Run中，需要为服务账号绑定对应的GKE集群权限：

• 创建或使用现有服务账号，为其添加container.deployments.delete、container.pods.delete等针对目标Namespace的权限
• 部署Cloud Run服务时指定该服务账号：gcloud run deploy ... --service-account=your-service-account@project.iam.gserviceaccount.com

3. 本地验证镜像 #

先在本地测试镜像是否能正常启动并监听5900端口：

docker run -p 5900:5900 -e NAMESPACE=nginx gcr.io/alealferez-project-1/kubedoom:v5

使用VNC客户端连接localhost:5900，确认服务正常运行，排除镜像本身的问题。

内容的提问来源于stack exchange，提问作者Alejandro Alferez Handszer