How would you diagnose and troubleshoot a Pod that is continuously crashing or failing to start in Kubernetes?

A systematic approach is crucial when a Kubernetes Pod is continuously crashing or failing to start. The primary goal is to identify the root cause, which can range from misconfigurations to resource issues or application-level bugs. Start by examining the Pod’s status and events, then delve into logs and YAML definitions. This methodical investigation allows for efficient problem isolation and resolution.

Common Causes of Pod Failure

Pod failures can stem from several common issues. `ImagePullBackOff` indicates the image cannot be pulled, perhaps due to a wrong name, tag, or registry authentication. `CrashLoopBackOff` signifies the container is starting, crashing, and restarting repeatedly, often due to an application error, an incorrect entrypoint, or an unhandled exception. `OOMKilled` (Out Of Memory Killed) means the container exceeded its memory limit. Other causes include volume mounting issues, failed liveness or readiness probes, network policy blocks, or incorrect security context settings.

Best practice

Always begin your diagnosis with `kubectl get pod ` to see its current status, then `kubectl describe pod ` to retrieve detailed information, including events, container statuses, and resource allocations. The events section is particularly vital, often pointing directly to issues like image pull failures, OOM conditions, or scheduler problems. Next, use `kubectl logs -c ` (or without `-c` if only one container) to check the application logs, including logs from previous container instances if it crashed, using `–previous`.

Edge case interviewers probe for

Interviewers might ask about less obvious scenarios. For instance, an `Init Container` failing will prevent the main application container from ever starting, leading to a persistent `Init:CrashLoopBackOff`. Another edge case is a Pod stuck in `Pending` due to insufficient resources (CPU, memory, or specific nodes labeled for taint tolerations) in the cluster, or a PersistentVolumeClaim failing to bind a volume. Network policies preventing necessary communication can also manifest as application-level timeouts, rather than immediate crashes.

Common mistake

A common mistake is to jump directly to modifying the deployment without fully understanding the root cause. This often leads to trial-and-error changes that can worsen the situation or mask the actual problem. Another mistake is neglecting to check the Pod’s events (`kubectl describe pod`) or previous container logs (`kubectl logs –previous`) which often contain critical clues. Overlooking resource limits and requests in the Pod’s definition, especially for memory, is another frequent oversight leading to `OOMKilled` issues.

What the interviewer is checking

The interviewer is checking your systematic problem-solving approach, your familiarity with `kubectl` commands for diagnosis, and your understanding of core Kubernetes concepts like Pod lifecycle, container states, resource management, and common failure modes. They want to see if you can methodically narrow down possibilities, interpret diagnostic output, and propose effective solutions. Your ability to think under pressure and demonstrate practical operational skills is key.