OpenShift Online 3.9中应用路由失效问题求助

Hey there, let’s work through figuring out why your 5-month-old app suddenly stopped serving requests over the weekend. Since it was running fine before, we’ll focus on checking recent state changes, resource issues, and component connections in your OpenShift environment:

1. Verify Your Pod’s Health #

First, check if your pod is actually running and healthy:

• Run oc get pods to see the pod’s status. Look for Running in the STATUS column, and check the RESTARTS count—if it’s high, the pod is crashing repeatedly.
• If the pod isn’t running, pull its logs with oc logs <your-pod-name> to look for error messages (like missing dependencies, configuration issues, or runtime crashes).
• Use oc describe pod <your-pod-name> to review recent events—this will show if the pod was evicted, failed to pull an image, or hit resource limits.

2. Check Deployment/StatefulSet Consistency #

Since your app uses a single pod, confirm your deployment is in a ready state:

• Run oc get deployments (or oc get statefulsets if you’re using one) and make sure the DESIRED and READY counts match.
• If they don’t, run oc describe deployment <your-deployment-name> to check for rollout failures, invalid configuration updates, or scaling issues.

3. Validate Route ↔ Service ↔ Pod Connections #

The error suggests the endpoint isn’t serving requests, so let’s confirm the routing chain is intact:

• Route to Service: Run oc describe route <your-route-name> and check the Spec: To: field—ensure it points to your app’s service.
• Service to Pod: Check if your service is linked to the pod via endpoints:
  • Run oc get endpoints <your-service-name>—you should see your pod’s IP listed here.
  • If endpoints are empty, verify the service’s label selector matches the pod’s labels. Run oc describe service <your-service-name> to get the selector, then oc describe pod <your-pod-name> to check the pod’s labels.

4. Rule Out TLS/Edge Termination Issues #

While the error message doesn’t point directly to TLS, it’s worth checking since you’re using Edge termination:

• Run oc get route <your-route-name> and review the TLS section to confirm the termination type is edge and the certificate hasn’t expired.
• If you’re using a custom certificate, double-check its validity; if using OpenShift’s default, it should auto-renew, but it’s still worth verifying.

5. Check OpenShift Starter Resource Limits #

Starter environments have strict resource quotas—your app might have hit a limit and been evicted:

• Run oc get quota to see your current resource usage vs. allocated limits (CPU, memory, pod count).
• Run oc get limits to check per-container resource constraints.
• Look for OutOfMemoryKilled or Evicted events in oc describe pod <your-pod-name>—this means the pod exceeded its memory/cpu limits.

6. Review Recent Changes #

Even if no one intentionally modified the app, automated processes or accidental changes could have caused issues:

• Check deployment history with oc rollout history deployment <your-deployment-name> to see if a new rollout was triggered over the weekend.
• Run oc get events --sort-by='.metadata.creationTimestamp' to view recent cluster events—look for unexpected restarts, scaling, or configuration changes.

7. Test Internal Cluster Access #

To isolate whether the issue is with the Route or the app itself:

• Spin up a temporary curl pod to test access to your service from inside the cluster:

  oc run -it --rm --image=curlimages/curl curl-test -- curl <your-service-name>:<service-port>
  

• If this works, the problem is with your Route or external networking. If it fails, the issue is with your pod or service.

Once you narrow down the root cause (e.g., crashing pod, missing endpoints, resource limits), you can target the fix—for example, adjusting resource requests, fixing a configuration error, or rolling back a bad deployment.

内容的提问来源于stack exchange，提问作者Abacus