Investigate and Recover a Bad Image Rollout · Kubernetes
Create a controlled image failure, inspect the evidence, and restore the previous template.
Learn with progress and quizzesFirst, understand the concepts.
Rollouts and availability
A Deployment replaces Pods using a new template. Some old Pods can remain during a rolling update, so a new Pod failure does not automatically mean every replica is down.
Use events to narrow the cause
A bad image tag fails during image pulling. Read describe and Events rather than relying only on a broad Pending status.
Rollback scope
rollout undo restores a previous Pod template. It does not restore external database changes or independently edited ConfigMaps.
Why it matters
A bad image tag can create replacement Pods that cannot pull their image. Existing replicas serving traffic do not mean the new rollout succeeded.
Work through the reasoning
Identify the failed revision
Connect the current template, new ReplicaSet, and new Pod events. A rollout timeout means the requested condition was not reached in time; events help explain why.
Verify recovery
After undo, verify rollout completion, image, and Ready count. Recheck the originally failing condition instead of relying only on a command’s success message.
Learn by doing.
Use kubectl, a personal practice Kubernetes cluster, and a POSIX shell. Check the context and use the dedicated namespace in each lab. Cleanup deletes all resources in that practice namespace.
Change a healthy Deployment to an intentionally nonexistent image tag. The rollout wait failure is expected; inspect events and then undo the change.
kubectl config current-context
kubectl create namespace study-k8s-recovery --dry-run=client -o yaml | kubectl apply -f -
kubectl create deployment web --image=nginx:stable-alpine -n study-k8s-recovery --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout status deployment/web -n study-k8s-recovery --timeout=120s
kubectl set image deployment/web nginx=nginx:study-missing-tag-20261117 -n study-k8s-recovery
kubectl rollout status deployment/web -n study-k8s-recovery --timeout=30s || true
kubectl get pods -n study-k8s-recovery
kubectl get events -n study-k8s-recovery --sort-by=.metadata.creationTimestamp
kubectl rollout undo deployment/web -n study-k8s-recovery
kubectl rollout status deployment/web -n study-k8s-recovery --timeout=120s
kubectl get deployment web -n study-k8s-recovery -o jsonpath='{.spec.template.spec.containers[0].image}'How to check your result
Find an image lookup failure in Events and confirm readiness returns after rollback. Network failures or registry limits can also break pulls, so distinguish their messages. The recovered image should be nginx:stable-alpine.
Clean up practice resources
Run after checking the result. Confirm these resources belong to your practice run and contain no data you need to keep.
kubectl delete namespace study-k8s-recoveryApply what you learned
You restore the old revision but the app still uses new configuration. Why?
Reveal the reasoning after trying
Rollback restores the Pod template, not historical content in a same-name ConfigMap. Define a separate strategy for versioning images and configuration together.
Ready to check your understanding?
What does rollout undo primarily restore?
- All external data
- A previous Pod template
- Every Secret in the cluster
- The node operating system
Reveal the reasoning after trying
A previous Pod template
Deployment rollback restores a previous Pod template, not all external application state.
Official documentation · Investigate and Recover a Bad Image Rollout