Kubernetes how-to
kubectl logs: Read Pod Logs, Crashes and Selectors
kubectl logs prints what a container wrote to stdout and stderr. It is the first command most people run when a pod misbehaves, and it is also the command that gives the most confusing output: empty results, the wrong container, or logs from a run that ended minutes ago. This guide covers the flags you need and the cases where the command has nothing to show.
By Muhammad Soliman, Founder at SeaGit
·
Key takeaways
- kubectl logs reads one container at a time. In a multi-container pod, add -c NAME or --all-containers.
- Use --previous (-p) to read the logs of the last terminated instance of a container. This is how you see why a CrashLoopBackOff pod died.
- Narrow the output with --since, --since-time and --tail before you pipe it to grep. A busy service can produce megabytes in an hour.
- A label selector with -l, or deployment/NAME, reads several pods in one command. Set --max-log-requests if the selector matches a lot of pods.
The basic form, and the flags you will use most
The shape of the command is kubectl logs POD, optionally followed by flags. If the pod has one container, that is all you need. The reference page lists the full set of options, but five cover almost every incident.
# Snapshot of the current container logs
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod
# Stream new lines as they arrive (Ctrl+C to stop)
kubectl logs -f my-api-7d9f8b6c5-x2k4q -n prod
# Last 200 lines, and only the past 15 minutes
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod --tail=200 --since=15m
# Prefix each line with the pod and container name
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod --prefix
# Add timestamps from the container runtime
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod --timestampsUse -f for a live view while you reproduce a bug. Use --tail and --since for the normal case, because the default is the whole retained log and that can be large.
Two more flags help with size. --limit-bytes caps the output in bytes, which matters when one line is enormous, such as a stack dump or a serialized payload. --timestamps adds the time the runtime recorded for each line, which lets you line up the application log with an event in kubectl get events. Both work with -f as well as with a snapshot.
Time filters are relative to now, so --since=15m on a snapshot means the last fifteen minutes at the moment you press enter. If you need a fixed window for a ticket, record the start with --since-time in RFC3339 form, and keep the output in a file.
Multi-container pods, deployments and label selectors
A pod with a sidecar, an init-style helper or a proxy has more than one container. If you omit -c, kubectl returns an error that lists the container names. Name the one you want with -c, or read all of them with --all-containers.
Init containers keep logs too. Read one with -c NAME. Their output is often the only record of a failed migration or a dependency check that never passed, because the app container never started far enough to write anything of its own.
# Pick one container in a multi-container pod
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod -c app
# Every container in the pod
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod --all-containers=true
# One pod picked for a Deployment (kubectl chooses a replica)
kubectl logs deployment/my-api -n prod
# All pods behind the Deployment, with the pod name as prefix
kubectl logs deployment/my-api -n prod --all-pods=true --prefix
# Every pod with the label app=my-api, every container, streamed
kubectl logs -f -l app=my-api -n prod --all-containers=true --max-log-requests=10Read the replicas side by side
When a bug appears on one replica and not another, the selector form is the fastest way to see both. The --prefix flag puts the pod name in front of each line, so you can tell which replica said what. Keep --max-log-requests at a sensible number: each matching pod is one request against the API server and the kubelet.
- Selectors match on labels, not names. A label that is missing from one replica silently drops it from the output.
- deployment/NAME and statefulset/NAME resolve to pods through the workload, so they stop working if the selector changes.
- Logs from a pod that has been deleted are gone with it. Ship logs off the node if you need them after a rollout.
Crash loops: read the previous container with --previous
When a container exits, kubelet restarts it. The new container starts with an empty log stream, so a plain kubectl logs shows only the current run, which often has just the first few lines of start-up. The output that explains the crash belongs to the previous instance. The --previous flag returns that output.
# Logs of the last terminated instance of the app container
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod -c app --previous
# Same thing, short form
kubectl logs -p my-api-7d9f8b6c5-x2k4q -n prod -c app
# Confirm the exit code and reason before reading the log
kubectl describe pod my-api-7d9f8b6c5-x2k4q -n prod | sed -n '/Last State/,/Ready/p'If the container has never restarted, --previous fails with a message that no previous terminated container exists. That is the answer: the crash is not happening yet. For a full walk-through of the restart states, see the CrashLoopBackOff guide linked below.
Why kubectl logs returns nothing
Empty output is the most common complaint, and it usually has one of a handful of causes. Work through them in this order before you assume the application is silent.
- The container has not written anything yet. A fresh pod in Pending or ContainerCreating has no logs at all. Check kubectl describe pod for the events.
- The app writes to a file, not to stdout or stderr. kubectl logs reads only the streams the runtime captures. Either log to stdout, or read the file with kubectl exec.
- You are looking at the wrong container. The pod shows no output because the sidecar is silent. Use -c or --all-containers.
- The container restarted, and you are reading the new instance. Add --previous to read the one that died.
- The pod was deleted or evicted. The log lives with the pod object on the node, so it is gone with the pod.
- The pod runs a container that has no shell and no logger, such as a scratch image that only prints on errors. The silence may be correct.
Check the source of truth
# Which containers exist, and what state each is in
kubectl get pod my-api-7d9f8b6c5-x2k4q -n prod \
-o jsonpath='{range .status.containerStatuses[*]}{.name}{"\t"}{.state}{"\n"}{end}'
# Events explain scheduling, image pull and probe failures
kubectl get events -n prod --field-selector involvedObject.name=my-api-7d9f8b6c5-x2k4qPractical patterns for incidents
Three patterns cover most incidents. The first narrows a live problem to the last few minutes. The second searches across replicas. The third keeps a copy of the output, so the evidence survives the next restart.
# Errors in the last 10 minutes, across every replica
kubectl logs -l app=my-api -n prod --all-containers=true --since=10m --prefix \
| grep -iE 'error|panic|timeout' | head -50
# Save the previous crash for a ticket, with timestamps
kubectl logs my-api-7d9f8b6c5-x2k4q -n prod -c app --previous --timestamps \
> crash-$(date +%Y%m%d-%H%M).logkubectl logs is a window onto what the kubelet still holds. For anything you need after a rollout, a node replacement or an eviction, send container output to a log store. kubectl is the right tool for the live question and a poor one for the historical record.