Use Taints and Tolerations for Dedicated Nodes · Kubernetes
Permitting a node’s taint does not force placement on that node.
Learn with progress and quizzesFirst, understand the concepts.
Repel versus permit
A taint affects Pods without a matching toleration. A toleration alone does not direct placement; combine it with nodeSelector or required affinity to target a dedicated pool.
Three effects
NoSchedule restricts new scheduling without evicting existing Pods. PreferNoSchedule is a softer preference. NoExecute can evict existing untolerated Pods. This lab uses only NoSchedule.
Placement is not authorization
Taints are not security boundaries. Unrestricted tolerations can bypass intended separation. Production pools also need appropriate admission policies, permissions, and protected label design.
Why it matters
To keep ordinary work away from a GPU pool, use a taint and give eligible applications matching tolerations. Restricting those applications to the GPU pool requires a separate placement constraint.
Work through the reasoning
Compare exclusion with selection
Match Node key, value, and effect to the Pod toleration. Broad Exists rules may tolerate more than intended. This lab uses Equal with a specific key, value, and effect.
Observe existing versus new work
Make existing Ready before adding NoSchedule. Observe it remaining while blocked waits. NoExecute can also evict workloads and is not an interchangeable test.
Learn by doing.
Use kubectl, a personal practice Kubernetes cluster, and a POSIX shell. cert-manager and VPA labs require the components specified in each lesson. Node-changing labs require a dedicated disposable worker. Use the practice namespaces.
Use only a dedicated disposable practice worker. Set export STUDY_NODE=actual-worker-name and explicitly mark it with kubectl label node "$STUDY_NODE" study.greatjin.net/lab-node=true. The code verifies that marker and absence of its practice taint. Do not use shared production or control-plane nodes. Keep the same shell for cleanup.
(
set -eu
kubectl config current-context
kubectl get nodes -L kubernetes.io/hostname
: "${STUDY_NODE:?Set STUDY_NODE to a Ready schedulable practice worker name}"
study_host=$(kubectl get node "$STUDY_NODE" -o jsonpath='{.metadata.labels.kubernetes\.io/hostname}')
test -n "$study_host"
test "$(kubectl get node "$STUDY_NODE" -o jsonpath='{.metadata.labels.study\.greatjin\.net/lab-node}')" = true
study_existing=$(kubectl get node "$STUDY_NODE" -o jsonpath='{.spec.taints[?(@.key=="study.greatjin.net/dedicated")].key}')
test -z "$study_existing"
kubectl create namespace study-node-taints --dry-run=client -o yaml | kubectl apply -f -
kubectl apply -n study-node-taints -f - <<YAML
apiVersion: v1
kind: Pod
metadata:
name: existing
spec:
nodeSelector:
kubernetes.io/hostname: "$study_host"
containers:
- name: worker
image: busybox:1.36
command: ["sleep", "3600"]
YAML
kubectl wait -n study-node-taints --for=condition=Ready pod/existing --timeout=120s
kubectl taint node "$STUDY_NODE" study.greatjin.net/dedicated=practice:NoSchedule
kubectl apply -n study-node-taints -f - <<YAML
apiVersion: v1
kind: Pod
metadata:
name: blocked
spec:
nodeSelector:
kubernetes.io/hostname: "$study_host"
containers:
- name: worker
image: busybox:1.36
command: ["sleep", "3600"]
YAML
kubectl wait -n study-node-taints --for=condition=PodScheduled=false pod/blocked --timeout=60s
kubectl get pods -n study-node-taints -o wide
kubectl describe pod blocked -n study-node-taints
kubectl patch pod blocked -n study-node-taints --type=merge -p '{"spec":{"tolerations":[{"key":"study.greatjin.net/dedicated","operator":"Equal","value":"practice","effect":"NoSchedule"}]}}'
kubectl wait -n study-node-taints --for=condition=Ready pod/blocked --timeout=120s
kubectl get pods -n study-node-taints -o wide
)How to check your result
After NoSchedule, existing should remain Running while blocked is Pending. Adding the matching toleration makes blocked Ready on the selected worker. Even after a failed run, remove the practice taint with cleanup; do not remove unrelated taint keys.
Clean up practice resources
Run after checking the result. Confirm these resources belong to your practice run and contain no data you need to keep.
: "${STUDY_NODE:?Use the same practice worker name as the lab}"
kubectl taint node "$STUDY_NODE" study.greatjin.net/dedicated:NoSchedule-
kubectl label node "$STUDY_NODE" study.greatjin.net/lab-node-
kubectl delete namespace study-node-taintsApply what you learned
A Pod has the toleration but remains Pending. What else should you inspect?
Reveal the reasoning after trying
Check other untolerated taints, nodeSelector and required affinity, resource requests, PVC binding, and unschedulable nodes alongside Events. Permitting one exclusion does not satisfy every scheduling requirement.
Ready to check your understanding?
How do you require placement on a dedicated node while allowing its NoSchedule taint?
- Set only a toleration
- Combine nodeSelector or required affinity with a matching toleration
- Give the Pod the same name as the node
- Add only PreferNoSchedule
Reveal the reasoning after trying
Combine nodeSelector or required affinity with a matching toleration
The placement constraint restricts eligible nodes, while the toleration permits their taint. Neither replaces the other.
Official documentation · Use Taints and Tolerations for Dedicated Nodes