Skip to content

Admission Controllers and Webhooks

First PublishedByAtif Alam

After authentication and authorization, many API requests pass through admission — a chain of plugins that can mutate and/or validate objects before they are persisted in etcd.

Mutating vs validating

Section titled “Mutating vs validating”
PhaseRunsTypical examples
MutatingFirstDefault fields, inject sidecars, set labels, rewrite images
ValidatingAfter mutations settlePolicy-as-code, schema checks, security constraints

Both phases can be implemented as dynamic admission webhooks (MutatingWebhookConfiguration, ValidatingWebhookConfiguration).

failurePolicy and timeouts

Section titled “failurePolicy and timeouts”
  • failurePolicy: Fail — if the webhook is unreachable or times out, the request is rejected. Safest for strict security, but a misconfigured webhook can brick cluster operations (including scheduling control loops if they hit the API in ways that trigger admission).
  • failurePolicy: Ignore — failures allow the request through. Reduces outage risk from the webhook, but weakens guarantees — use only when you explicitly accept that tradeoff.

Timeouts matter: slow webhooks add latency to every matching request; at scale, they can overload the API server goroutine budget. Keep webhook logic O(1) on object size and avoid synchronous network calls where possible.

Debugging slow or failing webhooks

Section titled “Debugging slow or failing webhooks”
  1. Check API server logs for webhook timeouts and TLS errors.
  2. kubectl get validatingwebhookconfiguration,mutatingwebhookconfiguration — verify CABundle, service endpoints, and namespace of the webhook service.
  3. NetworkPolicy must allow apiserver → webhook service (path depends on topology).
  4. Temporarily scope webhooks with objectSelector / namespaceSelector to reduce blast radius while fixing.

Policy engines

Section titled “Policy engines”
  • Kyverno — Kubernetes-native policies; often mutate + validate in one chart.
  • OPA Gatekeeper — Rego policies with Constraint/ConstraintTemplate model.

For CI-time policy (Conftest, kubectl apply --dry-run=server), see Policy as code.

Image signature verification at admission

Section titled “Image signature verification at admission”

Pipeline signing (Cosign, attestations, SBOMs) proves what CI built. Admission-time verification proves what the cluster is about to run matches that policy — catching tampered registries, tag drift, and “wrong digest” deploys after merge.

Typical patterns:

  • Kyverno verifyImages rules — check signatures against expected keys or Sigstore CTlog policies.
  • Sigstore Policy Controller or similar validating webhooks — enforce that only signed images with required subjects/issuers are admitted.

Treat these webhooks like any other validating admission: tune timeouts, failurePolicy, and scope (namespaces, workload kinds) so a registry outage does not brick the whole cluster. For Cosign/SBOM depth in CI and compliance framing, see Compliance and Security scanning.

Section titled “Related”