如何通过Google Cloud Deploy多目标向Cloud Run部署不同镜像？

问题背景 #

通过Github Actions将多个镜像推送到Google Cloud Deploy单个版本时，使用Skaffold加载多个rawYaml文件会触发Cloud Build报错：

error: the manifest contains a KRM definition with kind: Service. As a result the manifest is assumed to define a Cloud Run Service. However, 2 KRM definitions were found, a manifest may only define one Cloud Run Service

需求是在单个多目标的每个子目标中部署不同的Cloud Run服务镜像。

错误原因 #

Cloud Deploy的Cloud Run部署器要求每个部署的manifest只能包含一个Service资源，但当前Skaffold会将所有services/下的yaml文件合并成一个manifest，导致单个manifest中存在多个Service，触发校验报错。

解决方案 #

方案一：拆分manifest并为子目标指定对应路径 #

1. 调整Skaffold配置

修改skaffold.yaml，明确声明每个服务的镜像和对应的rawYaml文件：

apiVersion: skaffold/v4beta6
kind: Config

artifacts:
  - image: service-name-1
    context: ./service1 # 替换为该镜像的实际构建上下文路径
  - image: service-name-2
    context: ./service2 # 替换为该镜像的实际构建上下文路径

manifests:
  rawYaml:
    - services/service-name-1.yaml
    - services/service-name-2.yaml

deploy:
  cloudrun: {}

2. 更新Cloud Deploy Target配置

在clouddeploy.yaml中，为每个子Target添加deployParameters指定对应服务的manifest路径：

apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: service-name-1
description: production service 1
run:
  location: projects/coolbeaver/locations/asia-southeast1
deployParameters:
  manifestPaths: ["services/service-name-1.yaml"]
---
apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: service-name-2
description: production service 2
run:
  location: projects/coolbeaver/locations/asia-southeast1
deployParameters:
  manifestPaths: ["services/service-name-2.yaml"]

3. 调整Github Actions流程

在Actions步骤中，用Skaffold渲染manifest并创建Cloud Deploy版本：

# 渲染manifest并替换镜像地址
skaffold render --default-repo gcr.io/your-project-id --output ./

# 创建Cloud Deploy版本，指定包含所有服务配置的目录
gcloud deploy releases create release-$(date +%Y%m%d%H%M%S) \
  --delivery-pipeline main-pipeline \
  --region asia-southeast1 \
  --source . \
  --manifests ./

方案二：使用Kustomize生成定制化manifest #

如果需要更灵活的配置管理，可以用Kustomize为每个服务生成独立的manifest：

1. 搭建Kustomize目录结构

kustomize/
├── base/
│   ├── service-template.yaml
│   └── kustomization.yaml
└── overlays/
    ├── service1/
    │   └── kustomization.yaml
    └── service2/
        └── kustomization.yaml

2. 编写基础模板和定制化配置

• base/service-template.yaml（通用Service模板）：

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: placeholder
spec:
  template:
    spec:
      containers:
        - image: placeholder-image

• base/kustomization.yaml：

resources:
  - service-template.yaml

• overlays/service1/kustomization.yaml（定制service-name-1）：

bases:
  - ../../base
images:
  - name: placeholder-image
    newName: service-name-1
patches:
  - patch: |-
      - op: replace
        path: /metadata/name
        value: service-name-1

• overlays/service2/kustomization.yaml（定制service-name-2）：

bases:
  - ../../base
images:
  - name: placeholder-image
    newName: service-name-2
patches:
  - patch: |-
      - op: replace
        path: /metadata/name
        value: service-name-2

3. 更新Skaffold配置

apiVersion: skaffold/v4beta6
kind: Config

artifacts:
  - image: service-name-1
    context: ./service1
  - image: service-name-2
    context: ./service2

manifests:
  kustomize:
    paths:
      - kustomize/overlays/service1
      - kustomize/overlays/service2

deploy:
  cloudrun: {}

4. 调整Github Actions流程

# 渲染所有Kustomize overlay生成manifest
skaffold render --default-repo gcr.io/your-project-id --output manifests/

# 创建Cloud Deploy版本
gcloud deploy releases create release-$(date +%Y%m%d%H%M%S) \
  --delivery-pipeline main-pipeline \
  --region asia-southeast1 \
  --source . \
  --manifests manifests/

核心原理 #

通过让每个Cloud Run子目标加载单独的、仅包含一个Service的manifest，规避Cloud Deploy的校验限制，同时利用单个Cloud Deploy版本包含所有服务配置，实现多目标的批量部署。

内容的提问来源于stack exchange，提问作者spicyzboss