K8s从私有Harbor拉取镜像失败：unexpected media type text/html

核心问题 #

在Kubernetes集群中从私有Harbor仓库拉取镜像时失败，容器事件显示核心错误：

failed to unpack image on snapshotter overlayfs: unexpected media type text/html

环境信息 #

• 所有节点：Ubuntu 20，containerd v1.6.17
• Docker节点：Docker v23.0.1，Harbor v2.5.5
• K8s节点：K8s v1.26.1

已执行配置操作 #

1. 所有节点配置hosts映射：

vim /etc/hosts
192.168.100.142 plzharbor.com

2. 生成CA证书并配置Harbor HTTPS：

openssl genrsa -out ca.key 4096

openssl req -x509 -new -nodes -sha512 -days 3650 \
 -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=plzharbor.com" \
 -key ca.key \
 -out ca.crt

openssl genrsa -out plzharbor.com.key 4096

openssl req -sha512 -new \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=plzharbor.com" \
    -key plzharbor.com.key \
    -out plzharbor.com.csr

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1=plzharbor.com
DNS.2=plzharbor
EOF

openssl x509 -req -sha512 -days 3650 \
    -extfile v3.ext \
    -CA ca.crt -CAkey ca.key -CAcreateserial \
    -in plzharbor.com.csr \
    -out plzharbor.com.crt

openssl x509 -inform PEM -in plzharbor.com.crt -out plzharbor.com.cert

3. 配置harbor.yml证书路径：

hostname: plzharbor.com
http:
  port: 80
https:
  port: 443
  # The path of cert and key files for nginx
  certificate: /data/cert/plzharbor.com.crt
  private_key: /data/cert/plzharbor.com.key

4. Docker节点配置证书后可正常登录Harbor并推送镜像plzharbor.com/k8mtest/k8m:v1.0，镜像Dockerfile：

FROM golang:1.20.1 as build

ENV GO111MODULE=on \
    CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64 \
    GOPROXY=https://goproxy.cn/,direct

WORKDIR /app
COPY . .
RUN go build -o main .

FROM golang:alpine

WORKDIR /
COPY --from=build /app/resource /resource
COPY --from=build /app/main main
EXPOSE 8181
CMD ["./main"]

5. K8s节点containerd私有仓库配置：

[plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."plzharbor.com"]
          [plugins."io.containerd.grpc.v1.cri".registry.configs."plzharbor.com".tls]
            insecure_skip_verify = false
            ca_file = "/etc/containerd/certs.d/plzharbor.com/ca.crt"
            cert_file = "/etc/containerd/certs.d/plzharbor.com/plzharbor.com.crt"
            key_file = "/etc/containerd/certs.d/plzharbor.com/plzharbor.com.key"
          [plugins."io.containerd.grpc.v1.cri".registry.configs."plzharbor.com".auth]
            username = "admin"
            password = "Harbor12345"
      [plugins."io.containerd.grpc.v1.cri".registry.headers]

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
        endpoint = ["https://registry.aliyuncs.com", "https://hub-mirror.c.163.com/"]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."plzharbor.com"]
        endpoint = ["https://plzharbor.com/"]
    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
      tls_cert_file = ""
      tls_key_file = ""

6. 创建K8s镜像拉取密钥并部署资源：
   密钥创建命令：

kubectl create secret docker-registry plz\
  --docker-server=plzharbor.com \
  --docker-username=admin \
  --docker-password=Harbor12345

Deployment与Service配置：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8m-deploy
spec:
  selector:
    matchLabels:
      app: k8m
  template:
    metadata:
      labels:
        app: k8m
    spec:
      containers:
        - image: plzharbor.com/k8mtest/k8m:v1.0
          name: k8m
          ports:
            - containerPort: 8181
          imagePullPolicy: IfNotPresent
      imagePullSecrets:
        - name: plz
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: svc-k8m
  name: svc-k8m
spec:
  selector:
    app: k8m
  type: NodePort
  ports:
    - port: 8181
      protocol: TCP
      targetPort: 8181
      nodePort: 30801

容器事件日志 #

Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  14s               default-scheduler  Successfully assigned default/k8m-deploy-574c7f7f9f-t59hm to w02
  Normal   BackOff    14s               kubelet            Back-off pulling image "plzharbor.com/k8mtest/k8m:v1.0"
  Warning  Failed     14s               kubelet            Error: ImagePullBackOff
  Normal   Pulling    3s (x2 over 14s)  kubelet            Pulling image "plzharbor.com/k8mtest/k8m:v1.0"
  Warning  Failed     3s (x2 over 14s)  kubelet            Failed to pull image "plzharbor.com/k8mtest/k8m:v1.0": rpc error: code = NotFound desc = failed to pull and unpack image "plzharbor.com/k8mtest/k8m:v1.0": failed to unpack image on snapshotter overlayfs: unexpected media type text/html for sha256:9e0ca048375f6acc79b1fb61008d69b89655b4d33efaa1f217a4d7c3b2c69a01: not found
  Warning  Failed     3s (x2 over 14s)  kubelet            Error: ErrImagePull

排查与解决方法 #

1. 错误原因分析 #

unexpected media type text/html说明containerd请求镜像时收到的不是OCI镜像层数据，而是HTML内容——大概率是Harbor返回了404页面、登录页面等网页内容，意味着镜像拉取请求的认证、路径或权限存在问题。

2. 具体修复步骤 #

（1）修正containerd镜像仓库端点

Harbor的镜像仓库API需要/v2路径，修改containerd配置中的mirror endpoint：

[plugins."io.containerd.grpc.v1.cri".registry.mirrors."plzharbor.com"]
    endpoint = ["https://plzharbor.com/v2"]

修改后重启containerd：

systemctl restart containerd

（2）验证证书配置正确性

确保K8s节点上的证书文件：

• /etc/containerd/certs.d/plzharbor.com/ca.crt是Harbor生成的CA根证书
• 证书权限设置为644，属主属组为root:root

chmod 644 /etc/containerd/certs.d/plzharbor.com/*.crt
chown root:root /etc/containerd/certs.d/plzharbor.com/*.crt

（3）重新创建镜像拉取密钥

删除现有密钥并重新创建（避免命令换行导致的配置错误）：

kubectl delete secret plz
kubectl create secret docker-registry plz \
  --docker-server=plzharbor.com \
  --docker-username=admin \
  --docker-password=Harbor12345

重启Deployment使密钥生效：

kubectl rollout restart deployment k8m-deploy

（4）直接在节点测试镜像拉取

使用ctr命令直接测试拉取，确认问题点：

ctr images pull --user admin:Harbor12345 plzharbor.com/k8mtest/k8m:v1.0

若失败，查看详细错误日志：

ctr images pull --user admin:Harbor12345 -v plzharbor.com/k8mtest/k8m:v1.0

根据错误提示进一步排查（如证书信任、网络连通性）。

（5）检查Harbor项目权限

确保Harbor中的k8mtest项目已设置为公开，或者admin用户对该项目拥有拉取权限。

内容的提问来源于stack exchange，提问作者Prodigalgal