使用自定义Docker镜像在Kubernetes部署Jenkins时遭遇CreateContainerError问题求助

Let's break down how to diagnose and fix this issue step by step—since your Jenkins container works locally but fails in Kubernetes, the problem is likely tied to differences between your local Docker environment and the K8s cluster setup.

Step 1: Get the Exact Error Details #

First, we need to stop guessing and get concrete error logs from Kubernetes. Run these commands to uncover what's going wrong:

1. List all pods to find your failed Jenkins pod:

   kubectl get pods
   

2. Get detailed info about the pod, focusing on the Events section at the bottom—this will tell you if the issue is image pull failure, permission denied, missing files, etc.:

   kubectl describe pod <your-jenkins-pod-name>
   

3. If the container tried to start but crashed, check the previous logs:

   kubectl logs <your-jenkins-pod-name> --previous
   

Step 2: Verify Image Consistency #

Your local build uses jenkins-casc:0.1, but your Kubernetes deployment references tpargmdiaz/jenkins-k8s:1.0. Double-check:

• Did you correctly tag and push the local image to your Docker registry? Run docker images locally to confirm the image exists, then verify it's pushed with:

  docker push tpargmdiaz/jenkins-k8s:1.0
  

• Can your Kubernetes nodes pull the image? Test this by logging into a cluster node and running:

  docker pull tpargmdiaz/jenkins-k8s:1.0
  

  If this fails, you might have registry authentication issues in K8s (you'll need to create an ImagePullSecret).

Step 3: Check File Permissions & Presence in the Image #

Your Dockerfile copies plugins.txt and jenkins-casc.yaml into the image, but let's confirm they're present and accessible:

• Locally, run this to inspect the image's contents:

  docker run --rm jenkins-casc:0.1 ls -l /usr/share/jenkins/plugins.txt /usr/local/jenkins-casc.yaml
  

• Ensure the jenkins user (UID 1000, the default user for this base image) has read access to these files. If permissions are wrong, add a RUN chown line to your Dockerfile:

  RUN chown jenkins:jenkins /usr/share/jenkins/plugins.txt /usr/local/jenkins-casc.yaml
  

Step 4: Fix Kubernetes Deployment Configuration Issues #

Looking at your Deployment YAML, there are a couple of things to adjust:

1. Container Port Mapping: Your Service exposes port 80, but Jenkins runs on port 8080 by default. Update your Service to target the correct port:

   ports:
   - name: http
     port: 80
     targetPort: 8080  # Add this line to map to Jenkins' default port
     type: ClusterIP
   

2. Resource Allocation: Jenkins needs sufficient memory/cpu to start. Add resource requests/limits to prevent OOM kills (a common pitfall even if it's not the current error):

   containers:
   - name: jenkins
     image: tpargmdiaz/jenkins-k8s:1.0
     resources:
       requests:
         memory: "512Mi"
         cpu: "500m"
       limits:
         memory: "1Gi"
         cpu: "1"
   

Step 5: Validate the Jenkins Configuration-as-Code File #

The jenkins-casc.yaml you downloaded via curl might have syntax errors or cluster-specific references. Locally, test if the image starts without volume mounts (to mimic the K8s environment):

docker run --rm -p 8080:8080 jenkins-casc:0.1

If this fails locally, the issue is with the image itself—not Kubernetes. Check the logs here to debug the CAS config.

Common Pitfalls to Watch For #

• Local Volume Overrides: Your docker-compose.yml mounts local files over the ones in the image, so your local test isn't identical to the image you pushed. Always test the image directly (without mounts) before pushing to the registry.
• Environment Variables: Your Dockerfile sets JAVA_OPTS and CASC_JENKINS_CONFIG, which are included in the image—no need to redefine them in Kubernetes unless you want to override them.

Once you get the exact error from kubectl describe pod, you'll be able to zero in on the fix quickly.

内容的提问来源于stack exchange，提问作者Maury Diaz