OKD中使用OpenStack Cinder创建PV遇认证及端口占用问题

一、Cinder PV 创建认证失败问题 #

环境信息 #

[centos@svc ~]$ oc version
Client Version: 4.8.11
Server Version: 4.12.0-0.okd-2023-03-18-084815
Kubernetes Version: v1.25.0-2786+eab9cc98fe4c00-dirty

错误日志 #

[centos@svc ~]$ oc create -f cinder-persistanvolume.yaml 
Error from server (Forbidden): error when creating "cinder-persistanvolume.yaml": persistentvolumes "pv01" is forbidden: error querying Cinder volume db044f6c-3420-4586-9e27-6e54268c994b: unable to initialize cinder client for region: RegionOne, err: cloud provider is not initialized: cannot initialize cloud provider using data from the secret: You must provide a password to authenticate

相关配置 #

• PV定义文件cinder-persistanvolume.yaml：

apiVersion: "v1"
kind: "PersistentVolume"
metadata:
  name: "pv01" 
spec:
  capacity:
    storage: "10Gi" 
  accessModes:
    - "ReadWriteOnce"
  cinder: 
    fsType: "ext3" 
    volumeID: "db044f6c-3420-4586-9e27-6e54268c994b" 

• 存储类信息：

[centos@svc ~]$ oc get storageclass
NAME                     PROVISIONER                RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
standard-csi (default)   cinder.csi.openstack.org   Delete          WaitForFirstConsumer   true                   9d

• OpenStack认证配置openrc：

#!/usr/bin/env bash

export OS_AUTH_TYPE=v3applicationcredential
export OS_AUTH_URL=...
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME="RegionOne"
export OS_INTERFACE=...
export OS_APPLICATION_CREDENTIAL_ID=...
export OS_APPLICATION_CREDENTIAL_SECRET=...

• cloud.conf配置：

[Global]
auth-url = ...
application-credential-id = ...
application-credential-secret= ...
region = RegionOne

解决方案 #

1. 配置集群Cloud Provider Secret
   OKD不会读取本地环境变量，需要将cloud.conf存储到kube-system命名空间的Secret中：

   oc create secret generic cloud-config --from-file=cloud.conf -n kube-system
   

   确保Secret中的配置与openrc完全一致，检查application-credential-id、application-credential-secret和auth-url的拼写与完整性。

2. 使用CSI类型PV定义
   当前默认存储类为CSI驱动cinder.csi.openstack.org，传统cinder类型PV在OKD 4.12中兼容性较差，改为CSI格式的PV：

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: pv01
   spec:
     capacity:
       storage: 10Gi
     accessModes:
       - ReadWriteOnce
     csi:
       driver: cinder.csi.openstack.org
       volumeHandle: "db044f6c-3420-4586-9e27-6e54268c994b"
       fsType: ext3
   

3. 验证OpenStack连通性
   在集群节点上测试Cinder API是否正常：

   source openrc
   openstack volume show db044f6c-3420-4586-9e27-6e54268c994b
   

   若能返回卷信息，说明认证有效，问题集中在集群侧Secret配置。

二、openstack-cloud-controller-manager 端口冲突问题 #

错误日志 #

error: failed to create listener: failed to listen on 127.0.0.1:10258: listen tcp 127.0.0.1:10258: bind: address already in use

Pod状态 #

[centos@svc cloud-provider-openstack]$ kubectl get pods -n kube-system -w
NAME                                       READY   STATUS             RESTARTS        AGE
csi-cinder-nodeplugin-2c96w                3/3     Running            0               34m
csi-cinder-nodeplugin-8hmpq                3/3     Running            0               34m
csi-cinder-nodeplugin-8sl7p                3/3     Running            0               34m
csi-cinder-nodeplugin-9qwf2                3/3     Running            0               34m
csi-cinder-nodeplugin-bpqp4                3/3     Running            0               34m
csi-cinder-nodeplugin-qmcqx                3/3     Running            0               34m
openstack-cloud-controller-manager-4l9gk   1/1     Running            0               5m31s
openstack-cloud-controller-manager-blthh   0/1     CrashLoopBackOff   5 (2m36s ago)   5m31s
openstack-cloud-controller-manager-rpmxg   0/1     Error              6 (3m2s ago)    5m31s

解决方案 #

1. 端口冲突原因
   端口10258是OKD默认kube-controller-manager的健康检查端口，手动部署的openstack-cloud-controller-manager会与原生组件抢占端口。

2. 修改控制器监听端口
   编辑openstack-cloud-controller-manager-ds.yaml，修改启动参数更换端口（例如改为10259）：

   args:
     - --cloud-config=/etc/cloud/cloud.conf
     - --cloud-provider=openstack
     - --leader-elect=true
     - --port=10259
     - --healthz-port=10259
   

   重新应用DaemonSet：

   oc apply -f openstack-cloud-controller-manager-ds.yaml -n kube-system
   

3. 检查集群原生组件集成
   OKD 4.x通常已集成OpenStack云提供商组件，无需手动部署openstack-cloud-controller-manager。检查原生kube-controller-manager的启动参数：

   oc describe pod kube-controller-manager-<node-name> -n openshift-kube-controller-manager | grep Args
   

   若已包含--cloud-provider=openstack和--cloud-config参数，删除手动部署的DaemonSet：

   oc delete ds openstack-cloud-controller-manager -n kube-system
   

内容的提问来源于stack exchange，提问作者Norhther