OpenShift中MinIO启用TLS后上传报403 Forbidden问题求助

我在OpenShift集群部署了基于MinIO存储的Laravel应用，HTTP环境下上传图片正常，但启用TLS（HTTPS）后持续收到403 Forbidden错误。

已执行的证书配置操作 #

• 从证书提供商获取证书文件：private.pem、minio.crt和minio.ca-bundle
• 将minio.crt与minio.ca-bundle合并为public.crt，执行命令创建OpenShift TLS Secret：

  oc create secret tls minio-tls --key private.pem --cert public.crt
  

• 将minio.crt重命名为public.crt，与private.pem放入/home/myuser/certs_minio/目录，创建ConfigMap：

  oc create configmap minio-certs --from-file=/home/myuser/certs_minio/
  

相关配置信息 #

Deployment配置 #

kind: Deployment
apiVersion: apps/v1
metadata:
  name: minio
  labels:
    app: minio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio    
    spec:
      volumes:
      - name: sail2-minio
        persistentVolumeClaim:
          claimName: sail2-minio
      - name: cert-volume
        configMap:
          defaultMode: 420
          name: minio-certs
      - name: ca-certs-volume
        configMap:
          defaultMode: 420
          name: openshift-service-ca.crt
      containers:
        - name: minio
          image: 'minio/minio:latest'
          args:
            - minio
            - server
            - /data/minio
            - '--certs-dir'
            - /.minio/certs
            - '--console-address'
            - ':8900'
          livenessProbe:
            httpGet:
              path: /minio/health/live
              port: 9000
              scheme: HTTP
            initialDelaySeconds: 120
            timeoutSeconds: 1
            periodSeconds: 20
            successThreshold: 1
            failureThreshold: 3
          env:
            - name: MINIO_ROOT_PASSWORD
              value: XXXX
            - name: MINIO_ROOT_USER
              value: YYYY
          ports:
            - containerPort: 9000
              protocol: TCP
            - containerPort: 8900
              protocol: TCP
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /data/minio
              name: sail2-minio
            - mountPath: /.minio/certs
              name: cert-volume
            - mountPath: /.minio/certs/CAs
              name: ca-certs-volume

Service配置 #

kind: Service
apiVersion: v1
metadata:
  name: minio
  labels:
    app: minio
spec:
  ports:
    - name: '9000'
      protocol: TCP
      port: 9000
      targetPort: 9000
    - name: '8900'
      protocol: TCP
      port: 8900
      targetPort: 8900
  selector:
    app: minio

Ingress配置 #

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: minio
  labels:
    app: minio
spec:
  tls:
    - hosts:
        - minio.my-server.com
      secretName: minio-tls
  rules:
    - host: minio.my-server.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: minio
                port:
                  number: 9000
    - host: admin-minio.my-server.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: minio
                port:
                  number: 8900

MinIO Route信息 #

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  labels:
    app: minio
  name: minio-6fztw
spec:
  host: minio.my-server.com
  path: /
  port:
    targetPort: "9000"
  tls:
    certificate: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----
      -----BEGIN CERTIFICATE-----
      YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
      -----END CERTIFICATE-----
      -----BEGIN CERTIFICATE-----
      ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
      -----END CERTIFICATE-----
    insecureEdgeTerminationPolicy: Redirect
    key: |
      -----BEGIN PRIVATE KEY-----
      AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
      -----END PRIVATE KEY-----
    termination: edge
  to:
    kind: Service
    name: minio
    weight: 100
  wildcardPolicy: None

测试及日志信息 #

MinIO Pod日志 #

Warning: Default parity set to 0. This can lead to data loss.
MinIO Object Storage Server
Copyright: 2015-2022 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2022-11-26T22-43-32Z (go1.19.3 linux/amd64)
Status: 1 Online, 0 Offline.
API: http://10.129.3.246:9000 http://127.0.0.1:9000
Console: http://10.129.3.246:8900 http://127.0.0.1:8900
Documentation: https://min.io/docs/minio/linux/index.html
You are running an older version of MinIO released 5 days ago
Update: Run `mc admin update`

集群内Pod的curl测试结果 #

curl minio.my-server.com -IL
HTTP/1.1 302 Found
content-length: 0
location: https://minio.my-server.com/
cache-control: no-cache

HTTP/2 400 
accept-ranges: bytes
content-length: 261
content-type: application/xml
server: MinIO
vary: Origin
date: Sat, 03 Dec 2022 00:01:39 GMT
set-cookie: 7d9780cddf96cdea5da436eeb53224e8=6143925dc959f1e842f292b797867446; path=/; HttpOnly; Secure; SameSite=None

Liveness Probe的curl测试结果 #

HTTP/1.1 302 Found
content-length: 0
location: https://minio.my-server.com/minio/health/live
cache-control: no-cache

HTTP/2 200 
accept-ranges: bytes
content-length: 0
content-security-policy: block-all-mixed-content
server: MinIO
strict-transport-security: max-age=31536000; includeSubDomains
vary: Origin
x-amz-request-id: 172D4EA033B0C73A
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
date: Sat, 03 Dec 2022 14:24:46 GMT
set-cookie: 7d9780cddf96cdea5da436eeb53224e8=2f4032867d91284df8fab35936da3c68; path=/; HttpOnly; Secure; SameSite=None
cache-control: private

排查建议 #

1. 解决TLS终止模式冲突
   当前Route使用edge终止（OpenShift处理TLS，转发HTTP到MinIO），但MinIO启动时指定了--certs-dir，会强制要求客户端用HTTPS连接，导致转发的HTTP请求被拒绝。

   • 方案一：修改Route的termination为passthrough，让TLS直接透传到MinIO服务端，由MinIO处理证书验证。
   • 方案二：移除MinIO启动参数中的--certs-dir，让MinIO以HTTP模式运行，完全由OpenShift路由处理TLS终止。

2. 修正MinIO证书文件名
   MinIO默认要求私钥文件名为private.key，当前ConfigMap中的私钥是private.pem，需重命名后重新创建ConfigMap：

   mv private.pem private.key
   oc delete configmap minio-certs
   oc create configmap minio-certs --from-file=/home/myuser/certs_minio/
   

3. 更新Laravel的MinIO配置
   检查config/filesystems.php中的MinIO配置，确保endpoint已改为https://minio.my-server.com，并确认access key和secret key正确，没有硬编码HTTP地址。

4. 配置MinIO CORS规则
   使用MinIO控制台或mc工具添加允许Laravel应用域名的CORS规则，例如：

   mc config host add minio https://minio.my-server.com YYYY XXXX
   mc admin config set minio cors.rule1.allowed_origins="https://your-laravel-domain.com"
   mc admin config set minio cors.rule1.allowed_methods="PUT,GET,POST,DELETE,OPTIONS"
   mc admin service restart minio
   

5. 调整Liveness Probe协议
   将Deployment中Liveness Probe的scheme改为HTTPS，避免探针因HTTP跳转导致异常：

   livenessProbe:
     httpGet:
       path: /minio/health/live
       port: 9000
       scheme: HTTPS
   

内容的提问来源于stack exchange，提问作者ChoosenOne