配置Pod模板Security Context后预填充MySQL镜像无法启动

我制作了一个包含预填充Schema的MySQL镜像，原本的Jenkins Pipeline和Pod配置可以正常运行，但为工作区添加动态PVC并设置Pod级SecurityContext后，MySQL容器启动失败，报错权限拒绝访问binlog.index。

相关配置文件 #

Dockerfile #

FROM mysql:8.0.31 as builder

# That file does the DB initialization but also runs mysql daemon, by removing the last line it will only init
RUN ["sed", "-i", "s/exec \"$@\"/echo \"not running $@\"/", "/usr/local/bin/docker-entrypoint.sh"]

# needed for intialization
ENV MYSQL_ROOT_PASSWORD=test

COPY ./sql-scripts /docker-entrypoint-initdb.d/

# Need to change the datadir to something else that /var/lib/mysql because the parent docker file defines it as a volume.
# https://docs.docker.com/engine/reference/builder/#volume :
#       Changing the volume from within the Dockerfile: If any build steps change the data within the volume after
#       it has been declared, those changes will be discarded.
RUN ["/usr/local/bin/docker-entrypoint.sh", "mysqld", "--datadir", "/initialized-db"]

FROM mysql:8.0.31

COPY --from=builder /initialized-db /var/lib/mysql

原始Pod模板YAML #

apiVersion: v1
kind: Pod
metadata:
  labels:
    label: 'backend'
spec:
  shareProcessNamespace: true
  containers:
    - name: "maven"
      image: maven:3.6.3-openjdk-11
      resources:
        requests:
          memory: "2Gi"
          cpu: "2"
        limits:
          memory: "10Gi"
          cpu: "10"
      command: [ sleep ]
      args: [ 1h ]
      securityContext:
        capabilities:
          add:
            - SYS_PTRACE
    - name: mysql
      image: myDockerRegistry/mysql8-integration-test:v5
      env:
        - name: MYSQL_USER
          value: test
        - name: MYSQL_PASSWORD
          value: test
        - name: MYSQL_ROOT_PASSWORD
          value: test
      securityContext:
        capabilities:
          add:
            - SYS_PTRACE

原始Pipeline配置 #

pipeline {
    agent {
        kubernetes {
            yaml libraryResource('pod-templates/backend.yaml')
        }
    }
    stages { ... }
}

修改后的配置 #

为工作区添加动态PVC：

workspaceVolume dynamicPVC(accessModes: 'ReadWriteOnce',requestsSize: "10Gi", storageClassName: 'premium-rwo')

同时在Pod模板中添加Pod级SecurityContext：

spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000

错误日志 #

2022-11-03 09:33:25+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.31-1.el8 started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
2022-11-03T09:33:25.839933Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2022-11-03T09:33:25.842508Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31) starting as process 13
2022-11-03T09:33:25.845263Z 0 [Warning] [MY-010122] [Server] One can only use the --user switch if running as root
mysqld: File './binlog.index' not found (OS errno 13 - Permission denied)
2022-11-03T09:33:25.845867Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-11-03T09:33:25.846078Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.31)  MySQL Community Server - GPL.

Jenkins生成的最终Pod YAML #

apiVersion: v1
kind: Pod
metadata:
  annotations:
    buildUrl: >-
      http://jenkins.jenkins.svc.cluster.local:8080/job/LegacyProjects/job/my-project/job/k8s-test/79/
    runUrl: job/LegacyProjects/job/my-project/job/k8s-test/79/
  labels:
    label: backend
    jenkins/jenkins-jenkins-agent: 'true'
    jenkins/label-digest: 4581eadfdfcb3d0141b8e8727b53b2ff9a3575ec
    jenkins/label: LegacyProjects_my-project_k8s-test_79-xgtxd
  name: my-project-k8s-test-79-xgtxd-2xw2r-8wj64
  namespace: jenkins
spec:
  containers:
    - args:
        - 1h
      command:
        - sleep
      image: 'maven:3.6.3-openjdk-11'
      name: maven
      resources:
        limits:
          memory: 10Gi
          cpu: '10'
        requests:
          memory: 2Gi
          cpu: '2'
      securityContext:
        capabilities:
          add:
            - SYS_PTRACE
      volumeMounts:
        - mountPath: /home/jenkins/agent
          name: workspace-volume
          readOnly: false
    - env:
        - name: MYSQL_USER
          value: test
        - name: MYSQL_PASSWORD
          value: test
        - name: MYSQL_ROOT_PASSWORD
          value: test
      image: 'myDockerRegistry/mysql8-integration-test:v5'
      name: mysql
      securityContext:
        capabilities:
          add:
            - SYS_PTRACE
      volumeMounts:
        - mountPath: /home/jenkins/agent
          name: workspace-volume
          readOnly: false
    - env:
        - name: JENKINS_SECRET
          value: '********'
        - name: JENKINS_TUNNEL
          value: 'jenkins-agent.jenkins.svc.cluster.local:50000'
        - name: JENKINS_AGENT_NAME
          value: my-project-k8s-test-79-xgtxd-2xw2r-8wj64
        - name: JENKINS_NAME
          value: my-project-k8s-test-79-xgtxd-2xw2r-8wj64
        - name: JENKINS_AGENT_WORKDIR
          value: /home/jenkins/agent
        - name: JENKINS_URL
          value: 'http://jenkins.jenkins.svc.cluster.local:8080/'
      image: 'jenkins/inbound-agent:4.11-1-jdk11'
      name: jnlp
      resources:
        limits: {}
        requests:
          memory: 256Mi
          cpu: 100m
      volumeMounts:
        - mountPath: /home/jenkins/agent
          name: workspace-volume
          readOnly: false
  nodeSelector:
    kubernetes.io/os: linux
  restartPolicy: Never
  securityContext:
    fsGroup: 1000
    runAsGroup: 1000
    runAsUser: 1000
  shareProcessNamespace: true
  volumes:
    - name: workspace-volume
      persistentVolumeClaim:
        claimName: pvc-workspace-my-project-test-79-xgtxd-2xw2r-8wj64
        readOnly: false

解决方案 #

方案1：为MySQL容器单独设置SecurityContext，覆盖Pod级配置 #

MySQL官方镜像默认以root用户运行，Pod级的runAsUser:1000会强制容器以非root用户启动，导致无法访问/var/lib/mysql目录下的文件。可以在MySQL容器的securityContext中显式指定以root运行：

修改Pod模板中的MySQL容器配置：

- name: mysql
  image: myDockerRegistry/mysql8-integration-test:v5
  env:
    - name: MYSQL_USER
      value: test
    - name: MYSQL_PASSWORD
      value: test
    - name: MYSQL_ROOT_PASSWORD
      value: test
  securityContext:
    capabilities:
      add:
        - SYS_PTRACE
    runAsUser: 0  # 强制以root用户运行
    runAsGroup: 0

方案2：调整镜像中数据目录的权限 #

如果不想以root运行MySQL，可以修改Dockerfile，在构建阶段为/var/lib/mysql目录设置1000用户的读写权限：

更新Dockerfile的最后阶段：

FROM mysql:8.0.31

COPY --from=builder /initialized-db /var/lib/mysql
# 修改目录权限，让1000用户拥有访问权限
RUN chown -R 1000:1000 /var/lib/mysql

重新构建镜像后，容器以1000用户运行时就能正常访问数据文件。

方案3：限制Pod级SecurityContext的作用范围 #

可以将Pod级SecurityContext替换为仅在需要挂载PVC的容器（maven和jnlp）中设置runAsUser，不过Jenkins的kubernetes agent默认会把workspaceVolume挂载到所有容器，因此方案1或2更直接可靠。

内容的提问来源于stack exchange，提问作者FranAguiar