Minikube中MySQL Pod出现CrashLoopBackOff状态的排查求助

在Minikube中部署MySQL时，Pod出现CrashLoopBackOff状态，日志提示InnoDB redo log文件创建失败，数据文件损坏或未正常关闭。已删除Service、StatefulSet、PVC及Minikube内镜像和容器，但问题依旧。

已创建的配置文件如下：

secret.yaml #

apiVersion: v1
kind: Secret
metadata:
  name: mysql
type: Opaque
data:
  MYSQL_ROOT_PASSWORD: MTIzNA==

service.yaml #

apiVersion: v1
kind: Service
metadata:
  name: app-mysql
  labels:
    app: mysql
    app.kubernetes.io/name: mysql
spec:
  ports:
  - name: mysql
    port: 3306
  clusterIP: None
  selector:
    app: mysql

statefulset.yaml #

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
      app.kubernetes.io/name: mysql
  serviceName: mysql
  replicas: 1
  template:
    metadata:
      labels:
        app: mysql
        app.kubernetes.io/name: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.3.0
        envFrom:
          - secretRef:
              name: mysql
        ports:
        - name: mysql
          containerPort: 3306
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
        livenessProbe:
          tcpSocket:
            port: 3306
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
        readinessProbe:
          tcpSocket:
            port: 3306
          initialDelaySeconds: 5
          periodSeconds: 2
          timeoutSeconds: 1
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 2Gi

Pod日志 #

2024-02-26 14:59:34+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.3.0-1.el8 started.
2024-02-26 14:59:34+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-02-26 14:59:34+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.3.0-1.el8 started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
2024-02-26T14:59:35.016581Z 0 [System] [MY-015015] [Server] MySQL Server - start.
2024-02-26T14:59:35.321366Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.3.0) starting as process 1
2024-02-26T14:59:35.329645Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-02-26T14:59:35.433059Z 1 [ERROR] [MY-012960] [InnoDB] Cannot create redo log files because data files are corrupt or the database was not shut down cleanly after creating the data files.
2024-02-26T14:59:35.433177Z 1 [ERROR] [MY-012930] [InnoDB] Plugin initialization aborted with error Generic error.
2024-02-26T14:59:35.855906Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2024-02-26T14:59:35.856297Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2024-02-26T14:59:35.856366Z 0 [ERROR] [MY-010119] [Server] Aborting
2024-02-26T14:59:35.858561Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.3.0)  MySQL Community Server - GPL.
2024-02-26T14:59:35.858638Z 0 [System] [MY-015016] [Server] MySQL Server - end.

核心问题 #

删除PVC后，Minikube虚拟机本地存储卷（HostPath）中的损坏数据并未被清除，重新创建PVC时旧数据被再次挂载，导致InnoDB初始化失败。另外StatefulSet与Headless Service的关联配置存在错误。

具体修复步骤 #

1. 彻底重置Minikube环境
   执行以下命令清除Minikube虚拟机内所有残留数据：

   minikube stop
   minikube delete
   minikube start
   

2. 修正StatefulSet的Service关联
   StatefulSet的serviceName需与实际创建的Headless Service名称一致，修改statefulset.yaml中的对应配置：

   spec:
     serviceName: app-mysql # 原配置为mysql，改为实际的Service名称app-mysql
   

3. 优化探针配置（可选）
   TCP探针可能在MySQL完成初始化前触发误重启，建议改用MySQL命令探针：

   livenessProbe:
     exec:
       command: ["mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p$(MYSQL_ROOT_PASSWORD)"]
     initialDelaySeconds: 60
     periodSeconds: 10
   readinessProbe:
     exec:
       command: ["mysql", "-h", "localhost", "-u", "root", "-p$(MYSQL_ROOT_PASSWORD)", "-e", "SELECT 1"]
     initialDelaySeconds: 30
     periodSeconds: 5
   

4. 重新部署资源
   按顺序重新应用所有配置文件：

   kubectl apply -f secret.yaml
   kubectl apply -f service.yaml
   kubectl apply -f statefulset.yaml
   

内容的提问来源于stack exchange，提问作者fgrd4035