What is an admission controller, and what's the difference between a validating and mutating webhook?
Quick Answer
Admission controllers are the last stage of the API server's request pipeline (after authentication and authorization) — they can inspect, modify, or reject a request before it's persisted to etcd. **Mutating** admission webhooks run first and can modify the object (e.g., automatically injecting a sidecar container, or setting a default label); **validating** admission webhooks run after mutation and can only accept or reject the (possibly now-modified) object, never change it further — this ordering guarantees validation always sees the final, fully-mutated version of an object.
Detailed Answer
Where admission control fits in the request pipeline
Recall the API server's request pipeline (see the API server question): authentication (who are you) → authorization (are you allowed to do this action) → admission control (should this specific request actually be allowed/modified, given business/policy rules) → persist to etcd. Even a fully authenticated and authorized request can still be rejected or altered at the admission stage — this is where cluster-specific policy enforcement lives, distinct from the more general "is this identity allowed to do this kind of thing at all" question RBAC answers.
Built-in admission controllers
Kubernetes ships with several built-in admission controllers compiled into the API server (enabled via a startup flag) — examples include NamespaceLifecycle (prevents creating objects in a namespace that's being deleted), LimitRanger (enforces LimitRange defaults/constraints), ResourceQuota (enforces namespace resource quotas), and, notably, PodSecurity (implementing Pod Security Admission — see that question).
Custom admission via webhooks
Beyond the built-in set, the API server can call out to external webhook services for custom admission logic — this is how tools like OPA Gatekeeper, Kyverno, Istio's sidecar injector, and cert-manager all plug into the cluster's request pipeline without needing to be built into Kubernetes itself.
MutatingAdmissionWebhook — can modify the object
Incoming request: create a Pod
→ Mutating webhook (e.g., Istio's injector) intercepts it
→ Modifies the Pod spec to add an Envoy sidecar container
→ The MODIFIED Pod spec continues through the pipeline
A mutating webhook receives the incoming object and can return a modified version of it (typically expressed as a JSON patch) — this is exactly the mechanism a service mesh uses to automatically inject a sidecar proxy into every Pod without the Pod's original author needing to include it themselves, and how tools might automatically inject default resource requests/limits, labels, or annotations onto objects that don't specify them.
ValidatingAdmissionWebhook — can only accept or reject
Incoming request: create a Pod (now possibly already mutated above)
→ Validating webhook (e.g., a policy engine) checks it against custom rules
→ Either allows it through unchanged, or rejects it with an error
A validating webhook cannot modify the object at all — it can only inspect the (already fully mutated) object and return an allow/deny decision, optionally with an explanatory message shown back to whoever submitted the request. This is how organization-specific policies get enforced — e.g., "every Deployment must have resource limits set," "container images must come from our approved internal registry," "no Pod may run as root" (for organizations wanting rules beyond what the built-in Pod Security Standards cover).
Why mutating webhooks run before validating ones
This ordering is deliberate and important: mutation happens first, so that by the time validation runs, it's evaluating the final state of the object — including anything automatically added by mutating webhooks — rather than validating an intermediate, incomplete version that's about to change. If this order were reversed, a validating webhook might approve an object that a subsequent mutation then changes into something that would have failed validation, silently undermining the policy enforcement's whole purpose.
Why this matters operationally
Admission webhooks are themselves a critical-path dependency for the entire cluster's ability to create/modify objects — if a webhook service is down, misconfigured, or slow, it can (depending on its configured failurePolicy) either fail open (allow requests through, defeating its purpose) or fail closed (block all matching requests cluster-wide, including entirely legitimate ones, until the webhook service recovers). Deploying webhook services with high availability and sensible timeouts is a genuine, non-trivial operational responsibility for any cluster relying on them for critical policy enforcement.