如何将Minikube SSL证书有效期延长至10年并解决RBAC权限问题

问题背景 #

在隔离训练环境中，需要将AMI上Minikube集群的证书有效期设置为10年，避免证书过期导致集群无法启动。因kubeadm会增加AMI启动时间，故采用自定义脚本扩展证书。证书有效期已成功延长至约10年，但集群启动后API server出现RBAC权限错误，例如apiserver-kubelet-client用户被禁止对nodes/proxy执行create操作。

现有证书扩展脚本 #

# Extend Certificates
extend_certificate() {
    local base_path="$1"
    local node_name="$2"
    local days=3649
    cd "$base_path"

    # Map of COMMON_NAME to "KEY_NAME:CA_CERT:CA_KEY"
    declare -A map=(
        ["front-proxy-client"]="front-proxy-client.key:front-proxy-ca.crt:front-proxy-ca.key"
        ["apiserver-etcd-client"]="apiserver-etcd-client.key:etcd/ca.crt:etcd/ca.key"
        ["apiserver-kubelet-client"]="apiserver-kubelet-client.key:ca.crt:ca.key"
    )

    for COMMON_NAME in "${!map[@]}"; do
        IFS=":" read -r KEY_NAME CA_CERT CA_KEY <<< "${map[$COMMON_NAME]}"
        
        openssl11 req -new -key "$KEY_NAME" -out "$COMMON_NAME.csr" -subj "/CN=$COMMON_NAME"
        openssl11 x509 -req -in "$COMMON_NAME.csr" -CA "$CA_CERT" -CAkey "$CA_KEY" -CAcreateserial -out "${COMMON_NAME}_new.crt" -days $days
        mv "$COMMON_NAME.crt" "${COMMON_NAME}_old.crt"
        mv "${COMMON_NAME}_new.crt" "$COMMON_NAME.crt"
    done

    cd "$base_path/etcd"
    COMMON_NAME="kube-etcd-healthcheck-client"
    openssl11 req -new -key healthcheck-client.key -out healthcheck-client.csr -subj "/O=system:masters/CN=$COMMON_NAME"
    openssl11 x509 -req -in healthcheck-client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out healthcheck-client_new.crt -days $days
    mv healthcheck-client.crt healthcheck-client_old.crt
    mv healthcheck-client_new.crt healthcheck-client.crt

    COMMON_NAME="$node_name"
    for cert_name in peer server; do
        openssl11 req -new -key "$cert_name.key" -out "$cert_name.csr" -subj "/CN=$COMMON_NAME" -addext "subjectAltName = DNS:localhost, DNS:$node_name, IP:192.168.49.2, IP:127.0.0.1, IP:0:0:0:0:0:0:0:1"
        openssl11 x509 -req -in "$cert_name.csr" -CA ca.crt -CAkey ca.key -CAcreateserial -out "${cert_name}_new.crt" -days $days -extensions SAN -extfile <(printf "\n[SAN]\nsubjectAltName=DNS:localhost, DNS:$node_name, IP:192.168.49.2, IP:127.0.0.1, IP:0:0:0:0:0:0:0:1")
        mv "$cert_name.crt" "${cert_name}_old.crt"
        mv "${cert_name}_new.crt" "$cert_name.crt"
    done
}

extend_certificate "/var/lib/docker/volumes/sf-node-1/_data/lib/minikube/certs" "$NODE_NAME"
extend_certificate "/var/lib/docker/volumes/sfdev-node-1/_data/lib/minikube/certs" "$DEV_NODE_NAME"

出现的RBAC错误 #

++ kubectl get svc -n ingress-nginx
++ grep -oP '^ingress-nginx-.*controller\b'
++ awk 'NR==1{print $1}'
error: error upgrading connection: unable to upgrade connection: Forbidden (user=apiserver-kubelet-client, verb=create, resource=nodes, subresource=proxy)

证书有效期检查结果 #

[root@ip-xxx-xx-xx-xxx xxxxxx]$ kubeadm certs check-expiration --cert-dir /var/lib/docker/volumes/sf-node-1/_data/lib/minikube/certs
CERTIFICATE                         EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
!MISSING! admin.conf                                                                                 
apiserver                           Aug 19, 2033 14:21 UTC   9y              ca                      no      
apiserver-etcd-client               Aug 18, 2033 14:25 UTC   9y              etcd-ca                 no      
apiserver-kubelet-client            Aug 18, 2033 14:25 UTC   9y              ca                      no      
!MISSING! controller-manager.conf                                                                     
etcd-healthcheck-client             Aug 18, 2033 14:25 UTC   9y              etcd-ca                 no      
etcd-peer                           Aug 18, 2033 14:25 UTC   9y              etcd-ca                 no      
etcd-server                         Aug 18, 2033 14:25 UTC   9y              etcd-ca                 no      
front-proxy-client                  Aug 18, 2033 14:25 UTC   9y              front-proxy-ca          no      
!MISSING! scheduler.conf                                                                              

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Aug 19, 2033 14:21 UTC   9y              no      
etcd-ca                 Aug 19, 2033 14:21 UTC   9y              no      
front-proxy-ca          Aug 19, 2033 14:21 UTC   9y              no

解决方案 #

1. 修复apiserver-kubelet-client的RBAC绑定 #

apiserver-kubelet-client用户需要绑定到预设的ClusterRole以获取nodes/proxy的操作权限：

# 检查现有绑定是否存在
kubectl get clusterrolebinding | grep apiserver-kubelet-client

# 若不存在则创建绑定
kubectl create clusterrolebinding apiserver-kubelet-client-binding \
  --clusterrole=system:kube-apiserver-to-kubelet \
  --user=apiserver-kubelet-client

2. 验证权限配置 #

执行命令确认权限已生效：

kubectl auth can-i create nodes/proxy --user=apiserver-kubelet-client

返回yes则说明权限配置正确。

3. 重启API server组件 #

让RBAC配置生效：

# 通过Minikube ssh重启API server
minikube ssh -- sudo systemctl restart kube-apiserver

# 若API server运行在容器中，也可直接重启容器
minikube ssh -- sudo docker restart $(sudo docker ps | grep kube-apiserver | awk '{print $1}')

4. 优化证书生成脚本（避免后续问题） #

重新生成apiserver-kubelet-client证书时，需确保证书Subject包含system:masters组，修改脚本中对应CSR生成命令：
将脚本中apiserver-kubelet-client的CSR生成行替换为：

openssl11 req -new -key "$KEY_NAME" -out "$COMMON_NAME.csr" -subj "/O=system:masters/CN=$COMMON_NAME"

这样生成的证书会带有正确的组标识，避免RBAC识别异常。

内容的提问来源于stack exchange，提问作者Aedan Lawrence