配置OpenShift Hub主备集群时OADP存储地址配置异常

问题背景 #

正在配置2个本地部署的OpenShift Hub（ACM）集群主备模式，已安装OADP Operator，当前配置BackupStorageLocation（bsl）和DataProtectionApplication（dpa）自定义资源，计划将备份存储在基于ODF的本地S3兼容存储中。但BSL始终处于Unavailable状态，错误显示Velero尝试访问AWS S3公共地址，而非本地S3端点。

错误信息 #

执行oc describe backupStorageLocation -n openshift-adp dpa-sample-1时的报错：

Message:               BackupStorageLocation "dpa-sample-1" is unavailable: rpc error: code = Unknown desc = RequestError: send request failed
caused by: Get "https://velero-backups.s3.amazonaws.com/?delimiter=%2F&list-type=2&prefix=velero%2F": dial tcp: lookup velero-backups.s3.amazonaws.com on 172.30.X.Y:53: no such host

尝试添加s3ForcePathStyle和s3Url字段到DPA时的警告：

W0929 17:20:34.664738   19524 warnings.go:70] unknown field "spec.backupLocations[0].velero.s3ForcePathStyle"
W0929 17:20:34.722556   19524 warnings.go:70] unknown field "spec.backupLocations[0].velero.s3Url"

环境信息 #

oc version
Client Version: 4.9.0-202203251214.p0.ga646be5.assembly.stream-a646be5
Server Version: 4.14.31
Kubernetes Version: v1.27.14+7852426

本地S3端点验证：同一域内服务器可通过以下命令访问存储桶：

aws --endpoint-url https://ocs-storagecluster-cephobjectstore-openshift-storage.apps.node-hub-test02.opsh-cls.linux.play.local s3 ls

正确存储桶地址：https://ocs-storagecluster-cephobjectstore-openshift-storage.apps.node-hub-test02.opsh-cls.linux.play.local/velero-backups-6374f848-c87e-4368-9904-ed43375ad5ff

当前资源配置 #

DataProtectionApplication (dpa-sample) #

Name:         dpa-sample
Namespace:    openshift-adp
API Version:  oadp.openshift.io/v1alpha1
Kind:         DataProtectionApplication
Spec:
  Backup Locations:
    Name:  default
    Velero:
      Config:
        Profile:  default
        Region:   us-east-1
      Credential:
        Key:    cloud
        Name:   cloud-credentials
      Default:  true
      Object Storage:
        Bucket:  ocs-storagecluster-cephobjectstore-openshift-storage.apps.node-hub-test02.opsh-cls.linux.play.local
        Prefix:  velero
      Provider:  aws
  Configuration:
    Node Agent:
      Enable:         true
      Uploader Type:  restic
    Velero:
      Default Plugins:
        openshift
        aws
  Snapshot Locations:
    Velero:
      Config:
        Profile:  default
        Region:   us-east-1
      Provider:   aws
Status:
  Conditions:
    Last Transition Time:  2024-09-29T13:43:02Z
    Message:               Reconcile complete
    Reason:                Complete
    Status:                True
    Type:                  Reconciled

BackupStorageLocation 状态 #

oc get bsl -n openshift-adp
NAME           PHASE         LAST VALIDATED   AGE     DEFAULT
default        Unavailable   53s              20h     true
dpa-sample-1   Unavailable   23s              3d22h   true

dpa-sample-1 BSL详情 #

Name:         dpa-sample-1
Namespace:    openshift-adp
API Version:  velero.io/v1
Kind:         BackupStorageLocation
Spec:
  Config:
    Profile:  default
    Region:   us-east-1
  Credential:
    Key:    cloud
    Name:   cloud-credentials
  Default:  true
  Object Storage:
    Bucket:  velero-backups
    Prefix:  velero
  Provider:  aws
Status:
  Last Validation Time:  2024-09-30T07:11:46Z
  Message:               BackupStorageLocation "dpa-sample-1" is unavailable: rpc error: code = Unknown desc = RequestError: send request failed
caused by: Get "https://velero-backups.s3.amazonaws.com/?delimiter=%2F&list-type=2&prefix=velero%2F": dial tcp: lookup velero-backups.s3.amazonaws.com on 172.30.X.Y:53: no such host
  Phase:  Unavailable

解决方案 #

问题出在DPA的配置字段位置错误：s3Url和s3ForcePathStyle应该放在spec.backupLocations[0].velero.config下，而非直接在velero层级。

修改后的DPA YAML示例 #

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
  namespace: openshift-adp
spec:
  backupLocations:
    - name: default
      velero:
        config:
          profile: default
          region: us-east-1
          s3Url: "https://ocs-storagecluster-cephobjectstore-openshift-storage.apps.node-hub-test02.opsh-cls.linux.play.local"
          s3ForcePathStyle: "true"
        credential:
          key: cloud
          name: cloud-credentials
        default: true
        objectStorage:
          bucket: velero-backups-6374f848-c87e-4368-9904-ed43375ad5ff
          prefix: velero
        provider: aws
  configuration:
    nodeAgent:
      enable: true
      uploaderType: restic
    velero:
      defaultPlugins:
        - openshift
        - aws
  snapshotLocations:
    - velero:
        config:
          profile: default
          region: us-east-1
        provider: aws

关键修正点 #

1. s3Url和s3ForcePathStyle移至config字段：这两个参数是Velero AWS插件的配置项，必须放在spec.backupLocations[].velero.config下，而非直接作为velero的子字段。
2. 修正Bucket名称：DPA中objectStorage.bucket应填写实际的存储桶名称velero-backups-6374f848-c87e-4368-9904-ed43375ad5ff，而非S3端点地址。端点地址通过s3Url配置。
3. s3ForcePathStyle设为"true"：ODF（Ceph RGW）默认使用路径风格访问，需启用此参数避免虚拟主机模式的域名解析错误。

应用配置步骤 #

1. 备份当前DPA配置：

oc get dpa dpa-sample -n openshift-adp -o yaml > dpa-backup.yaml

2. 应用修改后的DPA配置：

oc apply -f modified-dpa.yaml -n openshift-adp

3. 验证BSL状态：

oc get bsl -n openshift-adp

等待几分钟后，BSL状态应变为Available。

验证存储连接 #

可以通过查看Velero pod日志确认是否成功连接到本地S3存储：

oc logs deploy/velero -n openshift-adp | grep -i "backup storage location"

内容的提问来源于stack exchange，提问作者Liav