Skip to content
CloudSecOps

tutorial

Kubernetes guardrails with OPA Gatekeeper

Writing the constraint is the easy part. An operational guide to running Gatekeeper as admission control you can afford to have fail: fail-open defaults, cold caches, audit blind spots, staged rollout, and the silent-drop bugs that make a green CI run mean nothing.

Setu Parimi63 min read

Gatekeeper is usually sold as a policy-language problem and operated as an availability problem. This guide is written against Gatekeeper v3.23.0 on Kubernetes 1.34–1.36, verified 2026-08-06. It covers what the webhook does when it is down, slow, or cold; how to promote a constraint without causing an incident; and two silent-drop failure modes that let a policy pass its own tests while enforcing nothing.

Scope, and what this does not cover

This is an operations guide with policy examples, not a policy cookbook with an operations appendix. It assumes you can read Rego well enough to follow a twelve-line rule. It does not teach Rego as a language.

Everything here is verified against:

ComponentVersion verifiedEvidence
Gatekeeperv3.23.0, published 2026-07-09Project Helm repository index
Embedded OPAv1.17.1Gatekeeper OPA-versions table
Kubernetes1.36, 1.35, 1.34 supportedKubernetes releases page
ConstraintTemplate APItemplates.gatekeeper.sh/v1 (storage)Shipped CRD manifest at tag v3.23.0
Constraint APIconstraints.gatekeeper.sh/v1beta1Shipped CRD manifest at tag v3.23.0
Kyverno (comparison only)v1.18.2, chart 3.8.2, 2026-07-10Kyverno Helm repository index

The policy examples were executed: compiled with opa check and run through gator verify using the gator CLI built from upstream tag v3.23.0. Fifteen cases pass. That proves the policies evaluate as claimed; it proves nothing about how they behave in your cluster.

Gatekeeper v3.24.0-beta.0 was published 2026-07-13. If you are reading this after v3.24 goes stable, re-check the deprecations in the failure-modes and rollout sections — one flag is already slated for removal.

Out of scope, deliberately:

  • Rego as a language reference. Enough syntax to read the examples, no more.
  • External data providers beyond stating their maturity and the latency consequence of putting a network call in the admission path.
  • Image signature and supply-chain verification. Adjacent problem, different article.
  • Runtime detection. Referenced only to mark the boundary where admission control stops being the right tool.
  • Benchmarks. gator bench produces latency percentiles. CloudSecOps has not run it and will not publish numbers it did not measure. The article describes the shape of the cost — serial mutating webhooks, parallel validating webhooks, a 3-second timeout, a 512Mi memory limit — and leaves measurement to you.
  • Cluster-level behaviour. Nothing here was tested on a live cluster. gator evaluates the constraint framework outside the API server; it does not exercise webhook delivery, TLS, the audit controller, or the data cache.
  • Vendor redistributions. This is written against upstream Gatekeeper. Azure Policy for AKS and Google's Policy Controller pin their own versions and lag upstream. Azure Policy for AKS additionally restricts template installation to templates delivered through Azure Policy definitions; Policy Controller supports custom ConstraintTemplates directly. Check what your distribution actually ships before applying any version-specific claim here.

What this article replaces

CloudSecOps published a multi-part OPA Gatekeeper guardrail series in early 2021, indexed at cloudsecops.com/opa-gatekeeper/ under the title "A Series of Blog Posts on using OPA Policies & Gatekeeper for Kubernetes Security", published 2021-02-16. The recovered index lists five guardrail categories — Pod Security, RBAC, NetworkPolicy, Storage Classes, and CI/CD — of which Storage Classes appears to have been announced but never published; no URLs for it exist in the Internet Archive's index of the domain.

The original posts were written by Setu Parimi with Vishal Pranav and Siddarth Tanna, against Gatekeeper release-3.1. Every legacy article URL tested on 2026-08-06 returned HTTP 522, a Cloudflare origin timeout, so the originals were recovered from Internet Archive snapshots rather than from the live site.

This is a new article under a current date, not a date-preserving migration. The subject matter and thesis carry forward; the code does not. The 2021 templates use templates.gatekeeper.sh/v1beta1 with openAPIV3Schema blocks that omit type: object, which the v1 API rejects as non-structural. One contains an unterminated string literal and does not compile. Another defines a helper rule named contains, a keyword in Rego v1. A third reuses one index variable across three collections and therefore only ever checks one port per ingress rule.

Both compilation claims were re-tested with OPA 1.19.0 rather than asserted from reading: the unterminated literal produces rego_parse_error: non-terminated string, and a rule named contains produces rego_parse_error: unexpected if keyword under Rego v1.

Publishing a defect analysis of your own back catalogue is uncomfortable, and it is the most useful thing this article contains — those bugs are the kind that pass code review, ship, and quietly fail to enforce anything. The same review found three total-bypass defects in this article's own replacement policies, before publication, by running them rather than reading them. Both sets are documented in the policy-examples section.

All 2021 posts linked to hands-on lab environments at katacoda.com/cloudsecops/.... Katacoda was shut down in 2022 and those links are dead. They are not reproduced here.

What changed between 2021 and 2026

Four structural changes make the original series unrunnable rather than merely dated.

PodSecurityPolicy is gone. PSP was deprecated in v1.21 and removed in v1.25. Several 2021 Pod Security guardrails were framed as PSP equivalents. Pod Security Admission, stable since v1.25, covers much of that ground with three namespace labels (pod-security.kubernetes.io/enforce, -audit, -warn) and three levels (privileged, baseline, restricted). If you are still running PSP you are on an unsupported minor, and that is the higher-priority finding.

The ConstraintTemplate API moved to v1 and requires a structural schema. Upstream is explicit: "Unlike past versions of ConstraintTemplate, v1 requires the Constraint schema section to be structural" and "One such requirement is that the type field be defined for each level of the schema." The shipped CRD serves v1, v1beta1 and v1alpha1 with v1 as storage, so old templates are not rejected — but anything written to v1 must carry type: at every level. Constraints are still constraints.gatekeeper.sh/v1beta1; there is no v1 constraint API. Teams that assume the template bump implied a constraint bump write manifests that never apply.

OPA went to 1.x, and Rego v1 is opt-in inside Gatekeeper. Gatekeeper v3.23.0 embeds OPA v1.17.1; v3.19.0 was the first release on OPA v1.x. The obvious inference — that Rego v1 syntax is now the default and old violation[{...}] { ... } rules break — is wrong in both halves. Upstream states: "Using Rego v1 syntax is opt-in, by default only Rego v0 is allowed." Rego v1 is enabled per template, in the newer code[] block, with an explicit version: "v1". The upstream policy library corroborates this: several current templates still ship v0 syntax in the legacy rego field with no version field at all.

In-tree CEL admission control went stable, for validation and mutation. ValidatingAdmissionPolicy has been stable since v1.30. MutatingAdmissionPolicy carries a FEATURE STATE: Kubernetes v1.36 [stable] banner and is enabled by default. The Kubernetes guidance, in full rather than in the half usually quoted: "use webhook admission control when you want an extensible way to declare or configure the logic. Use built-in CEL-based admission control when you want to declare simpler logic without the overhead of running a webhook server. The Kubernetes project recommends that you use CEL-based admission control when possible."

The middle sentence matters. The recommendation is not that webhooks are obsolete; it is that simple logic belongs in-tree, and configurable logic is what a webhook is for. Gatekeeper's parameterised constraint model is exactly the "configure the logic" case. This narrows what Gatekeeper is for without retiring it.

2021 assumptionTrue in 2026Consequence
PodSecurityPolicy is the baselineRemoved in v1.25; PSA is stablePod Security guardrails must be re-derived, not ported
templates.gatekeeper.sh/v1beta1 schemasv1 is storage and requires structural schemaOriginal templates fail validation on v1
Gatekeeper release-3.1v3.23.0Flags, CRDs and defaults all moved
Rego v0 is the languageOPA v1.17.1 embedded; v0 is a compatibility modeAuthor v1, opt in explicitly
Rego is the only engineCEL engine stable since v3.18 and takes precedenceEditing Rego in a dual-engine template can be a no-op
No in-tree policyVAP stable 1.30; MAP stable 1.36Simple validations should start in-tree
No test harnessgator beta since v3.11Untested policy is a choice, not a constraint
Pod-only matchingExpansionTemplate renders workload controllersDeployment-level rejection is now achievable
Hand-written host-path matcherk8spsphostfilesystem v1.1.2 upstreamPrefer the maintained implementation
Katacoda labsShut down 2022Links removed, not replaced

Where admission control sits, and where it does not

Admission runs after authentication and authorisation, in two phases: every mutating controller and webhook, then every validating controller and webhook. Mutating webhooks run in sequence and may be reinvoked. Validating webhooks run in parallel. Only after both phases does the object reach etcd.

Illustrative — the admission path for a Pod create, with Gatekeeper at both the mutating and validating hops. Timeouts are Gatekeeper's shipped defaults, not the Kubernetes defaults.

Figure 1: sequenceDiagram diagram

Four things follow, and each is a limit on what any constraint can ever do.

Reads bypass admission entirely. The Kubernetes documentation states it plainly: "Admission controllers do not (and cannot) block requests to read (get, watch or list) objects, because reads bypass the admission control layer." A constraint cannot stop anyone reading a Secret. That is RBAC's job, and no amount of policy-as-code substitutes for it.

Admission constrains creation, not behaviour. Gatekeeper inspects an API object. It does not watch a process. A Pod that passes every constraint at admission and then execs a shell, opens a raw socket, or writes to a mounted path is invisible to Gatekeeper. Runtime detection is a separate control with a separate failure model.

A NetworkPolicy constraint constrains the NetworkPolicy object, not the traffic. If the cluster's CNI does not implement NetworkPolicy — and several do not, or do so only partially — then a constraint that enforces the shape of NetworkPolicy manifests is enforcing the shape of a document nobody reads. Verify enforcement at the data plane before you invest in policy about the control plane.

Resources that predate the constraint are already there. Admission is evaluated on CREATE and UPDATE. A constraint applied on Tuesday says nothing about what was admitted on Monday until something updates it, or until audit runs. This is the strongest argument for Gatekeeper over ValidatingAdmissionPolicy: the audit controller re-evaluates existing objects, and in-tree VAP does not.

Gatekeeper's architecture in one pass

Illustrative — Gatekeeper component and data flow. The trust boundary is the gatekeeper-system namespace; write access to anything inside it is equivalent to control over enforcement.

Figure 2: flowchart diagram showing kube-apiserver, gatekeeper-system namespace, ConstraintTemplate, Generated Constraint CRD, Constraint, SyncSet or Config, in-memory data.inventory, Constraint status.violations

The two deployments

The shipped manifest runs gatekeeper-controller-manager at replicas 3 with --operation=webhook --operation=mutation-webhook, and gatekeeper-audit at replicas 1 with --operation=audit --operation=status --operation=mutation-status --operation=generate. Both carry resources.limits.memory: 512Mi and requests: {cpu: 100m, memory: 512Mi} — a memory limit with no CPU limit, the right shape for a latency-sensitive component — plus priorityClassName: system-cluster-critical. The webhook deployment has a PodDisruptionBudget with minAvailable: 1.

That is roughly 2GB of committed memory and four pods before you have written a single constraint — a real fraction of the budget on a small cluster, and one honest reason a small team should look at ValidatingAdmissionPolicy first.

Audit is a singleton by design. Overlapping runs cause a newer run to pre-empt the reporting of an older one, which surfaces as constraints with empty violation lists for no visible reason. Do not scale it.

The three webhooks

This is where the shipped manifest tells you something the documentation does not emphasise.

WebhookfailurePolicytimeoutSecondsOperations
validation.gatekeeper.shIgnore3CREATE, UPDATE plus a fixed subresource list
mutation.gatekeeper.shIgnore1CREATE, UPDATE
check-ignore-label.gatekeeper.shFail3CREATE, UPDATE on namespaces

Kubernetes defaults failurePolicy to Fail and timeoutSeconds to 10. Gatekeeper deliberately overrides both, and states why: "Currently Gatekeeper is defaulting to using Ignore for the constraint requests, which means constraints will not be enforced at admission time if the webhook is down or otherwise inaccessible. This is because we cannot know the operational details of the cluster Gatekeeper is running on and how that might affect webhook uptime."

The third webhook fails closed, on purpose: "The namespace label webhook defaults to Fail, this is to help ensure that policies preventing labels that bypass the webhook from being applied are enforced. Because this webhook only gets called for namespace modification requests, the impact of downtime is mitigated."

The practical consequence contradicts the common summary. In a stock, fail-open Gatekeeper install, if every Gatekeeper webhook pod is unavailable, namespace creates and updates fail. "Gatekeeper fails open" is about 90% true out of the box. The remaining 10% will find you during a cluster-wide incident, when someone is trying to create a namespace to stage a fix.

Both policy webhooks carry the same namespace selector:

Production-ready — verbatim from the shipped deploy/gatekeeper.yaml at tag v3.23.0. Reproduced to document the default; do not edit this by hand in a Helm-managed install.

namespaceSelector:
  matchExpressions:
    - key: admission.gatekeeper.sh/ignore
      operator: DoesNotExist
    - key: kubernetes.io/metadata.name
      operator: NotIn
      values: ["gatekeeper-system"]

Note what is not in that list. kube-system is not exempt by default. The only namespace the shipped manifest exempts is gatekeeper-system, via that selector and --exempt-namespace=gatekeeper-system on the controller. The Helm chart ships exemptNamespaces: []. The docs recommend exempting kube-system ("Exempting kube-system namespace is a good starting place"); the default install does not do it. A widely repeated assumption to the contrary has produced a lot of surprised operators.

The constraint framework

A ConstraintTemplate defines a schema and one or more policy implementations. Gatekeeper generates a CRD from it. A Constraint is an instance of that CRD, carrying spec.match, spec.parameters, and spec.enforcementAction.

Four enforcement points exist:

Enforcement pointWhat it isRegoCEL
validation.gatekeeper.shThe admission webhookYesYes
audit.gatekeeper.shThe audit controllerYesYes
gator.gatekeeper.shThe gator CLI, shift-leftYesYes
vap.k8s.ioGenerated in-tree ValidatingAdmissionPolicyNoYes

That last row is the whole reason to care about CEL inside Gatekeeper. A Rego-only template cannot be projected into an in-tree VAP; a CEL template can be, and Gatekeeper will generate the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding for you. VAP management is beta and enabled by default since v3.20, with --default-create-vap-for-templates and --default-create-vap-binding-for-constraints both defaulting to true. It requires Kubernetes 1.30 or later.

These enforcement points do not present identical inputs to your policy. That sounds like a detail. It is the subject of the first failure mode below.

The data cache

Referential constraints — anything that needs to know about objects other than the one being admitted — read from data.inventory, an in-memory replica of selected cluster state. The canonical example from upstream: "it is impossible to know if a label is unique across all pods and namespaces unless a ConstraintTemplate has access to all other pods and namespaces."

Two mechanisms populate it: Config.spec.sync.syncOnly (original, still alpha) and SyncSet (syncset.gatekeeper.sh/v1alpha1, since v3.15, recommended, multiple SyncSets union). Data is reachable at data.inventory.cluster and data.inventory.namespace.

Templates declare their data needs with a metadata.gatekeeper.sh/requires-sync-data annotation. That annotation is descriptive, not prescriptive — it documents what the template needs; it does not cause anything to be synced. The two drift, which is why gator sync test exists.

ConstraintTemplates and Constraints as they exist today

The code[] model and engine precedence

Modern templates put implementations in spec.targets[].code[], each with an engine (Rego or K8sNativeValidation) and a source. The legacy spec.targets[].rego field still works and is still used upstream. Two precedence rules, both documented, both capable of wasting an afternoon:

  1. "The legacy spec.targets[].rego field takes precedence over any Rego engine defined in spec.targets[].code[]."
  2. "The K8sNativeValidation (CEL) engine has higher priority than the Rego engine with no fallback mechanism."

There is no fallback. From the VAP documentation: "There is no fallback mechanism between engines, hence a logical/syntactical error in the policy logic is treated as violation depending on the enforcement action specified in the Constraint."

Combine those with the upstream library and you get a concrete trap, confirmed by reading the shipped templates for this article: k8spsphostfilesystem v1.1.2 and k8spsphostnetworkingports v1.1.5 both ship a K8sNativeValidation block and a Rego block. On v3.18 or later, the CEL block is what runs. Fork one and fix a bug in the Rego, and nothing changes in the cluster and nothing tells you. The two are not behaviourally identical either: the CEL branch short-circuits UPDATE inline, the Rego branch routes through a lib.exclude_update helper.

A related schema hazard: generateVAP does not appear in the ConstraintTemplate CRD's OpenAPI schema. It lives under spec.targets[].code[].source, marked x-kubernetes-preserve-unknown-fields: true. The API server accepts a misspelled generateVAPP: false without complaint and Gatekeeper does nothing with it. Typos there are silent.

Rego v0 and Rego v1

Rego v1 is per-template, and only under the newer code block:

Illustrative — the shape required to opt a template into Rego v1. import rego.v1 is not needed and not used.

targets:
  - target: admission.k8s.gatekeeper.sh
    code:
      - engine: Rego
        source:
          version: "v1"
          rego: |
            package k8sexample

            violation contains {"msg": msg} if {
              input.review.object.spec.hostNetwork
              msg := "hostNetwork is not permitted"
            }

Upstream: "Rego v1 syntax can only be used under targets[_].code[_].[engine: Rego].source with version: \"v1\". No need to add import rego.v1 to use rego v1 syntax."

CloudSecOps recommendation, not a documented requirement: author new templates in Rego v1 with the explicit version: "v1", even though v0 still works. OPA's own documentation describes --v0-compatible mode as "not recommended for most users", the migration direction is one-way, and opa fmt --v0-v1 exists specifically to rewrite v0 modules. Writing v0 in 2026 is accepting a migration you will have to do anyway, on someone else's schedule.

One thing you do not need to worry about: the Rego package name need not match the ConstraintTemplate's metadata.name. Tested here by installing a template named k8snamemismatch whose Rego declared package totallydifferentpackagename; the violation fired normally under gator v3.23.0. A mismatch is a readability problem, not a functional one.

Match semantics

spec.match on a constraint narrows scope by kinds, namespaces, excludedNamespaces, labelSelector, namespaceSelector, scope and name. Narrowing the match is the cheapest, most auditable exception mechanism Gatekeeper offers, and the one teams reach for last.

Failure modes

Almost every production problem with Gatekeeper is a consequence of running a self-hosted webhook in the critical path of the API server, and almost none of them are policy-language problems. The two exceptions are below, and both were found by running policy rather than reading it.

Silent rule drop: undefined terms, including in the message

Rego rule bodies are conjunctions: if any expression is undefined, the rule does not fire. That includes the expression that builds the rejection message.

The consequence: a violation message that references an optional constraint parameter silently disables the rule that produces it whenever that parameter is absent. The constraint is installed, its status is active, audit reports zero violations, and it enforces nothing.

Verified — reduced test case, run with OPA 1.19.0. The two rules differ only in how the message reads a parameter that the constraint did not set.

package probe

# Constraint omits `missingParam`. This rule produces NOTHING.
violation contains {"msg": msg} if {
  input.review.object.bad
  msg := sprintf("bad, allowed=%v", [input.parameters.missingParam])
}

# Same logic, defaulted parameter. This rule fires.
violation contains {"msg": msg} if {
  input.review.object.bad
  msg := sprintf("bad, allowed=%v", [object.get(input.parameters, "missingParam", [])])
}

Against input where .bad is true, the first rule returns an empty set and the second returns one violation. Nothing in Gatekeeper warns you about the first.

The mitigation is mechanical: bind every optional parameter through object.get with a default at the top of the package and reference the bound name everywhere else. Every policy here does that. A gator verify case that installs the constraint without its optional parameters catches the class.

Silent rule drop: input.constraint is not populated at every enforcement point

It is common advice to build a rejection message that names the constraint. Under gator v3.23.0, input.constraint is not populated. It is not merely missing a subfield — the reference itself is undefined, so even object.get(input.constraint, ["metadata", "name"], "fallback") is undefined, because object.get on an undefined first argument is undefined.

The result: a policy naming the constraint in its message evaluates correctly under opa eval and produces zero violations under gator test and gator verify. A CI job treating a green gator run as evidence will promote that constraint to deny believing it works.

Verified — behaviour under gator v3.23.0. The unguarded form yields no violations; the guarded form yields the fallback.

# Undefined under gator. The whole violation rule is dropped.
constraint_name := input.constraint.metadata.name

# Guarded. Complete-rule default supplies a value when the reference is undefined.
default constraint_label := "<constraint>"
constraint_label := input.constraint.metadata.name

Two conclusions. If you want the constraint name in a message, guard it with a default complete rule as above — object.get is not sufficient. Simpler: leave the constraint name out. Gatekeeper prefixes it automatically at admission, so it is redundant where it works and a landmine where it does not. Dropping it is what took the suites below from failing to passing.

The general lesson is larger than one field: the four enforcement points do not guarantee identical inputs. Test at the enforcement point you intend to rely on.

The trade at the centre of everything

Fail-open means a constraint that stops enforcing without telling you. Fail-closed means a control that can stop your cluster, with a trigger you do not control.

Kubernetes describes when the failure policy applies: network errors, timeouts, connection failures, non-2xx or malformed responses, serialisation failures, and, for mutating webhooks, undecodable patch types. Critically: "An explicit rejection, correctly transmitted, always denies the API request, regardless of the failurePolicy setting." Failure policy is about the webhook being unreachable, never about it saying no.

CloudSecOps position: fail-open is the correct default for most clusters and it is also a silent control failure. The recommendation holds only if you pair it with an alert on webhook unavailability. With failurePolicy: Ignore and no alert on gatekeeper_validation_request_count going to zero, you do not have a control — you have a control-shaped object that stops working at the moment something is going wrong in the cluster.

Flip to Fail only when all of these hold: the webhook deployment is spread across failure domains and has a PDB; every namespace involved in bootstrap and recovery is exempt; you have a tested break-glass runbook; and you have decided explicitly that an admission outage beats an unenforced policy for these constraints. Scoping matters — failurePolicy is a property of the webhook configuration, so it applies to every constraint that webhook evaluates. You cannot fail closed on one policy and open on another within one webhook.

The deadlock

Upstream states the scenario better than a paraphrase would:

"Imagine you delete every Node in your cluster. This will kill all running Gatekeeper servers, which means the webhook will fail. Because a request to add a Node is subject to admission validation, it cannot succeed until the webhook can serve. The webhook cannot serve until a Node is added."

The escape hatch: "it should always be possible to modify or delete the ValidatingWebhookConfiguration because Kubernetes does not make requests to edit webhook configurations subject to admission webhooks." With one caveat that catches GitOps shops: "If the existence of the webhook resource is enforced by some external process (such as an operator), that may interfere with the emergency recovery process." If Argo CD or Flux is set to auto-sync and self-heal the Gatekeeper manifests, your break-glass procedure will be reverted within the sync interval. Test that before you need it.

Upstream also lists the scope-limiting traps: "Exempting kube-system namespace is a good starting place, but what about cluster-scoped resources, like nodes? What about other potentially critical namespaces like istio-system? … Did you know that a ConfigMap is used as the locking resource for some Kubernetes leader elections?" A constraint validating every ConfigMap, on a fail-closed webhook, in a cluster where leader election writes a ConfigMap every few seconds, is a component that can wedge control-plane leadership.

The full table

This table is CloudSecOps' consolidation. Upstream documents these behaviours across at least five separate pages and does not group them by blast radius. The Evidence column states what each row rests on.

FailureTrigger and observable signalBlast radiusEvidenceMitigation
Silent rule drop, undefined termOptional parameter referenced without a default; audit shows zero violationsThat constraint, silentlyTested, OPA 1.19.0Bind parameters via object.get with defaults; negative test in CI
Silent rule drop, input.constraintMessage names the constraint; gator reports zero violationsThat constraint, at some enforcement pointsTested, gator v3.23.0Omit the constraint name, or guard with a default rule
Silent non-enforcementWebhook pods down; failurePolicy: Ignore; gatekeeper_validation_request_count flatEvery constraint, cluster-wideShipped manifestAlert on request count and pod readiness; treat as a sev
Namespace writes blockedSame trigger; check-ignore-label is Fail; kubectl create ns errorsNamespace CREATE/UPDATEShipped manifestKnow it exists; do not rely on "fails open" during recovery
Admission deadlockFail-closed webhook plus loss of all webhook pods; nothing schedulesCluster-wideUpstream docsDelete the ValidatingWebhookConfiguration; disable GitOps self-heal first
Timeout non-enforcementEvaluation exceeds 3s; validation duration p99 near 3sWhichever requests time outShipped manifestReduce constraint fan-out; --max-serving-threads; scale replicas
Cold-cache false admitReferential constraint evaluated before data.inventory warmsReferential constraints onlyDesign inference, untestedGate readiness; avoid deny on referential policies during rollouts
Constraint stuck in errorTemplate compile failure or missing CRD; gatekeeper_constraints{status="error"} above 0That constraint, silentlyUpstream docsAlert on the metric; gator verify in CI
Audit results vanishOverlapping audit runs pre-empt each otherReporting onlyUpstream docsRaise --audit-interval above observed run duration
Violations truncatedMore than constraintViolationsLimit (default 20)Reporting onlyUpstream docsRaise toward 500; beyond that risks the 1.5MB etcd object limit
Pre-existing non-complianceConstraint created after the resources; admission clean, audit dirtyWhatever already existedUpstream docsRead audit before promoting to deny
Non-compliant deletesDELETE not in the shipped webhook rulesDELETE requestsShipped manifestEnable DELETE if needed; accept that deletes are unauditable
Workload-controller bypassConstraint matches Pods; user creates a Deployment; FailedCreate on the ReplicaSetDeveloper experience, then non-enforcementUpstream docsExpansionTemplate, or match controller kinds too
Exemption as escalationAnyone who can label a namespaceAll policy in that namespaceShipped manifestGuard webhook plus RBAC on namespace labels; alert on the label
Engine-precedence no-opDual-engine template; Rego edited, CEL runs; behaviour unchanged after a "fix"That constraintLibrary templates readDiff both engines; gator test with CEL enabled
Scoped-action audit lossenforcementAction: scoped without naming audit; violations stop appearingReporting for that constraintUpstream docsName audit.gatekeeper.sh explicitly or use "*"
Mutation conflictTwo mutators target the same field; gatekeeper_mutator_conflicting_count above 0Unpredictable final objectUpstream docsAlert on the metric; keep mutators few and orthogonal

The cold cache, honestly labelled

This row is reasoning from the design, not a tested result and not a quoted upstream guarantee. It is the only such row in the table, which is why it is called out here as well as marked in the Evidence column — a reader skimming the table should not carry it away as fact.

The documentation establishes that referential constraints read data.inventory, that a watch manager populates the cache, and that readiness is gated by --readiness-retries. It does not state what a referential constraint evaluates to between pod start and cache warm.

The reasoned failure mode: "violation if no other object matches X" finds nothing in an empty inventory and admits; "violation if some other object matches X" also finds nothing and also admits. Either way an incomplete inventory biases toward admission — safe for availability, unsafe for enforcement. Treat it as a design-derived hypothesis: do not put referential constraints in deny if Gatekeeper restarts often, and check gatekeeper_sync_last_run_time after any rollout.

The delete hole

The shipped validating webhook registers only CREATE and UPDATE. DELETE and CONNECT are off by default. This directly affects any "deny pod exec" guardrail, which requires CONNECT — including one from the original 2021 series. Note that pods/exec does appear in the webhook's resource list; it is the operation, not the resource, that is missing.

If you turn DELETE on, understand the coverage gap upstream describes:

"Once a resource is deleted, it is gone. This means that non-compliant deletes cannot be audited via Gatekeeper's audit mechanism, and increases the importance of webhook-based enforcement."

And, in the same breath:

"Since the webhook fails open by default … it is possible for admission requests to have imperfect enforcement, which means some non-compliant deletes may still go through despite the policy. Normally such failures of webhook enforcement could be caught by audit, but deletes are not auditable."

Read together: DELETE enforcement is webhook-only, and the webhook fails open. There is no compensating control inside Gatekeeper. If deletion of a class of object matters to you, the durable control is RBAC plus the Kubernetes audit log, and Gatekeeper is at best a second layer.

Emergency recovery

Illustrative — validate the webhook configuration name against your own install before you need this. The name below is the upstream default.

# 1. Confirm the diagnosis: are the webhook pods actually unavailable?
kubectl -n gatekeeper-system get pods -l control-plane=controller-manager

# 2. If GitOps manages Gatekeeper, suspend reconciliation FIRST, or step 3
#    will be reverted within the sync interval.
#    (Argo CD example; adapt to your tooling.)
argocd app set gatekeeper --sync-policy none

# 3. Break glass. Webhook-config edits are not themselves subject to
#    admission webhooks, so this succeeds even in a deadlocked cluster.
kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io \
  gatekeeper-validating-webhook-configuration

# 4. Recover the cluster. Then restore enforcement by re-applying the
#    webhook configuration and re-enabling GitOps sync.

Pair this with detection. An attacker with write access to admissionregistration.k8s.io resources would run exactly step 3. The control is not to hide the command — it is documented upstream — but to alert on it. A Kubernetes audit-log rule on create, update and delete against validatingwebhookconfigurations and mutatingwebhookconfigurations, excluding your reconciler's service account, is a high-signal, low-volume detection. So is an alert on any namespace acquiring the admission.gatekeeper.sh/ignore label.

Namespace exemptions

Two exemption mechanisms exist and they are not equivalent.

--exempt-namespace is a process flag on the controller, combined with the admission.gatekeeper.sh/ignore label and the webhook's namespaceSelector. The API server does not call the webhook at all for that namespace, so nothing is evaluated or logged at admission time. Audit is a separate path: it still evaluates the namespace and still reports violations. Removing it from audit as well takes Config.spec.match.excludedNamespaces with processes: ["audit"]. Conflating the two is how an exempted namespace ends up looking clean at admission while audit has been reporting on it all along — or, worse, is silently dropped from both when only one was intended.

Config.spec.match.excludedNamespaces is evaluated inside Gatekeeper. The webhook is still called; Gatekeeper decides not to enforce. That costs latency but keeps the request path uniform and is changeable at runtime.

For availability-critical exemptions — the ones that exist so the cluster can recover — use the flag, because it survives Gatekeeper being broken. For policy-scope exemptions use the constraint's own match.

Why the label is guarded

Permission to label a namespace is permission to disable all Gatekeeper policy in it. That is why check-ignore-label.gatekeeper.sh exists and fails closed: a privilege-escalation boundary hiding inside a metadata field.

Treat patch on namespaces as a privileged verb and audit who holds it. In most clusters the set of principals that can label a namespace is much larger than the set anyone intended to be able to bypass admission control — namespace labels are a common target for platform automation, service meshes, and cost tooling.

What actually needs exempting

Beyond gatekeeper-system (exempt by default) and kube-system (recommended, not default), work through:

  • Cluster-scoped resources. Namespace exemption does nothing for Nodes, PersistentVolumes, ClusterRoles or CRDs. If a constraint matches a cluster-scoped kind, there is no namespace to exempt.
  • kube-node-lease. Node heartbeats. Kubernetes' own webhook guidance calls out node leases as something webhooks should not intercept.
  • Service-mesh control planes such as istio-system, which participate in the data path for everything else.
  • Leader-election ConfigMaps and Leases, wherever your controllers keep them.
  • Any namespace that must function during recovery, including whatever your CNI, CSI drivers and DNS run in.

The general rule: exempt anything whose failure prevents Gatekeeper from starting, and anything whose write path is hot enough that a 3-second timeout is a real risk.

Audit: what it does and does not tell you

Audit periodically re-evaluates existing cluster objects against every constraint and writes violations to constraint status. It is the mechanism that makes Gatekeeper more than an admission webhook.

SettingDefaultWhy it matters
--audit-interval60 seconds (0 disables)Must exceed observed audit duration or runs overlap
--constraint-violations-limit20Recommended ceiling 500; higher risks the 1.5MB etcd object limit
--audit-chunk-size500 (0 = infinite)Too small under throttling risks list-resumption token expiry
--audit-from-cachefalseReads live from the API server by default
audit replicas1Singleton by design

Constraint status carries three hard limits. Only the most recent run is reported, so status is a snapshot and not a ledger — a violation that disappears between runs left no record. The list is capped at constraintViolationsLimit, so a constraint showing exactly 20 violations is telling you "at least 20". And deletes are invisible, as above.

Three places to read violations, with different properties: constraint status is immediate and lossy; audit log events (audit_started, constraint_audited, violation_audited, audit_finished, correlated by audit_id) are durable if you ship logs and complete for the run; violation export via the Connection CRD is alpha. Audit log events include constraint_annotations, so annotations on a constraint become machine-readable in your log pipeline. Put the owning team there — the cheapest routing key you will ever add.

Policy examples, rewritten and tested

Every CloudSecOps policy below compiles under opa check and passes gator verify with the gator CLI at tag v3.23.0. Fifteen cases pass across three suites. The upstream library templates referenced are cited at the versions actually fetched on 2026-08-06.

What that proves: the rules evaluate as described, against the objects in the suites, at the gator enforcement point. What it does not prove: behaviour on a live cluster, under load, with a cold cache, at the admission or audit enforcement points, or against object shapes not in the suites. Run them in dryrun in your own cluster before trusting them.

Host paths

The 2021 template, verbatim from the Internet Archive snapshot:

Historical — CloudSecOps, 2021-01-25. Reproduced for analysis. Does not compile as printed and will not install on templates.gatekeeper.sh/v1.

violation[{"msg": msg, "details": {}}] {
input.review.object.kind == "Pod"
allowedpaths := input.parameters.paths
hostpath := input.review.object.spec.volumes[_].hostPath.path
not contains(allowedpaths,hostpath)
msg := sprintf("%v not in allowed paths,[hostpath])
}
contains(allowedpaths,hostpaths) {
hostpath == allowedpaths[_]
}

Eight defects:

  1. The sprintf string literal is never closed. Confirmed with OPA 1.19.0: rego_parse_error: non-terminated string.
  2. A user-defined rule named contains collides with an OPA built-in, and contains is a keyword in Rego v1. Confirmed: under --v1-compatible this yields rego_parse_error: unexpected if keyword.
  3. The helper is defined as contains(allowedpaths, hostpaths) but its body references the unbound outer name hostpath.
  4. Matching is exact, not prefix-based. Allowing /tmp does not allow /tmp/cache.
  5. No readOnly dimension. A writable mount of an allowed path passes.
  6. It matches Pods only. A Deployment creating a violating Pod is admitted, and the Pod is rejected later at ReplicaSet creation, surfacing as a FailedCreate event rather than a clear rejection at kubectl apply.
  7. It reads input.review.object.kind rather than input.review.kind.kind.
  8. The accompanying constraint is named psp-host-network-ports on a host-path policy — copy-paste residue.

Defect 4 is the interesting one, because upstream shows what correct looks like. k8spsphostfilesystem v1.1.2 does deliberate prefix-segment matching that "allows /foo, /foo/, /foo/bar etc., but disallows /fool, /etc/foo". Naive startswith matching would permit /fool when /foo is allowed — a real off-by-one in a path allow-list.

The right answer for most teams is the upstream template rather than any of this. It is versioned, tested with gator verify, ships both CEL and Rego, and carries metadata.gatekeeper.sh/bundle: "pod-security-baseline, pod-security-restricted". Pin the version.

If you need the original intent — host-path prefixes with a read-only requirement and an actionable message — here it is in current syntax:

Verified — compiles under opa check --v1-compatible (OPA 1.19.0) and passes gator verify (gator v3.23.0) against the three-case suite below. Not run on a live cluster. Prefer upstream k8spsphostfilesystem unless you need requireReadOnly.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sallowedhostpaths
  annotations:
    metadata.gatekeeper.sh/title: "Allowed host paths"
    cloudsecops.io/owner: "platform-security"
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedHostPaths
      validation:
        openAPIV3Schema:
          type: object
          properties:
            allowedPrefixes:
              type: array
              items:
                type: string
            requireReadOnly:
              type: boolean
  targets:
    - target: admission.k8s.gatekeeper.sh
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package k8sallowedhostpaths

              # Parameter defaults. Without these, a constraint that omits a
              # parameter makes the sprintf argument undefined, which drops the
              # whole violation rule and silently admits everything.
              allowed_prefixes := object.get(input.parameters, "allowedPrefixes", [])
              require_read_only := object.get(input.parameters, "requireReadOnly", false)

              input_volumes contains v if {
                v := input.review.object.spec.volumes[_]
                v.hostPath
              }

              input_containers contains c if { c := input.review.object.spec.containers[_] }
              input_containers contains c if { c := input.review.object.spec.initContainers[_] }
              input_containers contains c if { c := input.review.object.spec.ephemeralContainers[_] }

              # Segment-aware prefix match: "/foo" permits "/foo" and
              # "/foo/bar", but not "/fool".
              path_under(prefix, path) if {
                trimmed := trim_suffix(prefix, "/")
                path == trimmed
              }
              path_under(prefix, path) if {
                trimmed := trim_suffix(prefix, "/")
                startswith(path, concat("", [trimmed, "/"]))
              }
              path_allowed(path) if { path_under(allowed_prefixes[_], path) }

              violation contains {"msg": msg} if {
                volume := input_volumes[_]
                hostpath := volume.hostPath.path
                not path_allowed(hostpath)
                msg := sprintf("volume %q mounts hostPath %q, which is not under an allowed prefix. Allowed prefixes: %v. Owner: platform-security.",
                  [volume.name, hostpath, allowed_prefixes])
              }

              # A hostPath volume with no path cannot be evaluated. Fail closed.
              violation contains {"msg": msg} if {
                volume := input_volumes[_]
                not volume.hostPath.path
                msg := sprintf("volume %q declares a hostPath with no path and cannot be evaluated.",
                  [volume.name])
              }

              violation contains {"msg": msg} if {
                require_read_only
                volume := input_volumes[_]
                mount := input_containers[_].volumeMounts[_]
                mount.name == volume.name
                not mount.readOnly
                msg := sprintf("volume %q is a hostPath mount and must be mounted readOnly: true. Owner: platform-security.",
                  [volume.name])
              }

Note what the message does not contain: the constraint name. Gatekeeper prefixes that at admission, and referencing input.constraint is the silent-drop hazard above. Everything else a developer needs is there — the volume, the offending value, the permitted set, the owner.

RBAC wildcards

The 2021 series shipped two templates, K8sBlockWildCardVerb and K8sBlockWildCardSubjects, each with duplicated violation blocks because, quoting the post, "'OR' operator is not available in Rego and we have to deny if a wildcard is used either in 'Role' or 'ClusterRole'". The premise is wrong: Rego expresses OR by writing multiple rules with the same name, which is what the duplication was doing by accident. Matching on input.review.kind.kind removes the need entirely.

Four substantive defects:

  1. Only verbs: ["*"] was blocked. resources: ["*"] and apiGroups: ["*"] were not. The post's own "non-violation" example was apiGroups: ["*"], resources: ["pods"], verbs: [...], an over-broad rule presented as compliant.
  2. That same example listed "edit" as a verb. edit is not an RBAC verb; it is a ClusterRole name.
  3. K8sBlockWildCardSubjects blocked subjects[_].name == "*", which is not a meaningful RBAC construct. You cannot bind to a subject literally named *. The real escalation path is binding a powerful role to system:unauthenticated, system:anonymous or system:authenticated.
  4. Both used input.review.object.kind rather than input.review.kind.kind.

The upstream library has no RBAC-wildcard policy. It does have k8sdisallowanonymous v1.1.0, which blocks bindings to system:anonymous and system:unauthenticated — the control addressing the escalation path defect 3 missed — and k8sblockendpointeditdefaultrole v1.0.0, the mitigation for CVE-2021-25740.

Testing the first draft of the rewrite surfaced a fifth gap inspection had missed: a ClusterRole with nonResourceURLs: ["*"] produced no violation, because the field was not in the parameter enum and could not be added. That grants every non-resource endpoint, including /metrics and the pprof handlers. It is in the enum below.

Verified — passes gator verify (gator v3.23.0) against a five-case suite covering clean rules, verb wildcards, apiGroup wildcards, nonResourceURL wildcards, and the named exemption. Not run on a live cluster.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8snowildcardrbac
spec:
  crd:
    spec:
      names:
        kind: K8sNoWildcardRBAC
      validation:
        openAPIV3Schema:
          type: object
          properties:
            forbiddenWildcardFields:
              type: array
              items:
                type: string
                enum: ["verbs", "resources", "apiGroups", "nonResourceURLs"]
            allowedNames:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package k8snowildcardrbac

              forbidden_fields := object.get(input.parameters, "forbiddenWildcardFields",
                ["verbs", "resources", "apiGroups", "nonResourceURLs"])
              allowed_names := object.get(input.parameters, "allowedNames", [])
              kind := object.get(input.review, ["kind", "kind"], "<unknown>")
              name := object.get(input.review, ["object", "metadata", "name"], "<unnamed>")

              # Scope the exemption by kind. An unqualified name match lets any
              # Role in any namespace claim a name reserved for a bootstrap
              # ClusterRole and inherit the exemption with it.
              name_allowed if {
                kind == "ClusterRole"
                name in allowed_names
              }

              violation contains {"msg": msg} if {
                not name_allowed
                field := forbidden_fields[_]
                some i
                rule := input.review.object.rules[i]
                wildcard_entry(object.get(rule, field, []))
                msg := sprintf("%s %q: rules[%d].%s contains a wildcard. Wildcards are not permitted in %v. Enumerate the values explicitly, or request a time-bounded exception. Owner: platform-security.",
                  [kind, name, i, field, forbidden_fields])
              }

              # A bare "*" is not the only wildcard that grants everything.
              # nonResourceURLs match by prefix, so "/*" and "/apis/*" are
              # equally total; resources accept subresource globs like "*/exec".
              wildcard_entry(values) if { "*" in values }
              wildcard_entry(values) if {
                some v in values
                endswith(v, "*")
              }
              wildcard_entry(values) if {
                some v in values
                startswith(v, "*/")
              }

object.get(rule, field, []) with field bound to a string variable rather than a literal key was the construct least certain at draft time. It works; the suite exercises it across three field names.

Illustrative — the matching constraint. Kubernetes ships wildcards in cluster-admin and several aggregated roles by design.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sNoWildcardRBAC
metadata:
  name: no-wildcard-rbac
  annotations:
    cloudsecops.io/owner: "platform-security"
    cloudsecops.io/rollout-stage: "dryrun"
spec:
  enforcementAction: dryrun
  match:
    kinds:
      - apiGroups: ["rbac.authorization.k8s.io"]
        kinds: ["Role", "ClusterRole"]
  parameters:
    forbiddenWildcardFields: ["verbs", "resources", "apiGroups", "nonResourceURLs"]
    allowedNames:
      - "cluster-admin"

Two warnings. enforcementAction: dryrun is the most important line in it — this constraint applied straight to deny on an existing cluster will reject the next Helm chart that ships a ClusterRole, and the cluster is full of them.

And allowedNames is an exact-match list that will not scale. Kubernetes ships dozens of default ClusterRoles carrying wildcards, and enumerating them by hand produces a list that rots at every upgrade. Those roles are auto-reconciled and labelled kubernetes.io/bootstrapping: rbac-defaults; excluding them with a match.labelSelector on that label is more durable. Verify the label on your distribution first — this article did not test it, because that requires a cluster.

NetworkPolicy ingress ports

The 2021 rule, verbatim:

Historical — CloudSecOps, 2021-02-21. Reproduced for analysis. Contains an index-reuse defect described below.

violation [{"msg": msg}] {
  input.review.object.spec.podSelector.matchLabels.app == "webserver"
  some i
  allowed := input.parameters.ports
  provided := input.review.object.spec.ingress[i].ports[i].port
  provided != allowed[i]
  msg := "Applying Ingress rule for the provided port is not allowed"
}

The variable i indexes ingress[], ports[] and allowed[] simultaneously, so it only ever compares ingress rule n's port n against allowed port n. A NetworkPolicy with one ingress rule and two ports leaves the second unchecked. Set membership is expressed as positional inequality, so with ports: [443, 8443] an ingress rule allowing 8443 at index 0 is flagged while the genuinely unchecked port passes.

Two further problems. The target workload (app: webserver) is hardcoded in the Rego rather than parameterised, defeating the post's own reason for choosing Gatekeeper — "Gatekeeper lets us use the same template and just use different input parameters". And the constraint's metadata.name is k8sblockports while the Rego package is k8sblockedports; that mismatch was tested here and is harmless.

What testing the replacement found. The first version of the rewrite fixed the index reuse and stopped there. Run against a fuller set of NetworkPolicy shapes, it turned out to have three complete bypasses and one silent skip. All four are properties of the NetworkPolicy API, quoted here from the upstream type definitions:

  • An ingress rule with no ports field. "If this field is empty or missing, this rule matches all ports (traffic not restricted by port)." A rule with only a from clause opens everything and the naive policy reports nothing.
  • A port entry with no port field. "If this field is not provided, this matches all port names and numbers." Same outcome, one level down.
  • endPort. "endPort indicates that the range of ports from port to endPort if set, inclusive, should be allowed by the policy." A rule with port: 443, endPort: 9000 passes an allow-list check on port alone while opening 8,557 ports.
  • Named ports. port is an IntOrString. A named port resolves to a number at the Pod, not in the NetworkPolicy, so it cannot be checked here. Skipping it silently, as the first version did, turns it into an evasion. Failing closed on it is the honest behaviour. (A numerically spelled string port such as "8080" is not a concern: the API server validates string ports against IANA service-name rules, which require at least one letter.)

This is the same class of error the article criticises in the 2021 code — a rule that looks right, compiles, and under-enforces. Executing it against adversarial inputs is what caught it, which is the entire argument for gator verify. Fixing endPort surfaced a second-order problem: the obvious implementation emitted one violation per disallowed port, 8,556 for a single field, which would blow past constraintViolationsLimit toward the etcd object size limit. It reports once per field instead.

Verified — passes gator verify (gator v3.23.0) against a seven-case suite covering all four bypasses plus a clean policy and a deny-all policy. Not run on a live cluster.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8sallowedingressports
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedIngressPorts
      validation:
        openAPIV3Schema:
          type: object
          properties:
            allowedPorts:
              type: array
              items:
                type: integer
  targets:
    - target: admission.k8s.gatekeeper.sh
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package k8sallowedingressports

              allowed_ports := object.get(input.parameters, "allowedPorts", [])
              ns := object.get(input.review, ["object", "metadata", "namespace"], "<none>")
              nm := object.get(input.review, ["object", "metadata", "name"], "<unnamed>")

              port_allowed(p) if { p in allowed_ports }

              # 1. An ingress rule with no ports matches ALL ports.
              #    `not rule.ports` is undefined-only: it misses `ports: []`,
              #    which is present, empty, and permits every port just the same.
              violation contains {"msg": msg} if {
                some i
                rule := input.review.object.spec.ingress[i]
                count(object.get(rule, "ports", [])) == 0
                msg := sprintf("NetworkPolicy %s/%s: spec.ingress[%d] specifies no ports, which permits every port. Enumerate ports explicitly; allowed ports are %v.",
                  [ns, nm, i, allowed_ports])
              }

              # 2. A port entry with no port matches ALL ports and names.
              violation contains {"msg": msg} if {
                some i, j
                port := input.review.object.spec.ingress[i].ports[j]
                not port.port
                msg := sprintf("NetworkPolicy %s/%s: spec.ingress[%d].ports[%d] omits port, which permits every port. Allowed ports are %v.",
                  [ns, nm, i, j, allowed_ports])
              }

              # 3. Numeric port outside the allow-list.
              violation contains {"msg": msg} if {
                some i, j
                port := input.review.object.spec.ingress[i].ports[j]
                is_number(port.port)
                not port_allowed(port.port)
                msg := sprintf("NetworkPolicy %s/%s: spec.ingress[%d].ports[%d] allows port %v; allowed ports are %v.",
                  [ns, nm, i, j, port.port, allowed_ports])
              }

              # 4. endPort opens an inclusive range; every port in it must be
              #    allowed. Reported once per field, not once per port, so a
              #    wide range cannot flood status past constraintViolationsLimit.
              violation contains {"msg": msg} if {
                some i, j
                port := input.review.object.spec.ingress[i].ports[j]
                is_number(port.port)
                is_number(port.endPort)
                disallowed := [p | some p in numbers.range(port.port, port.endPort); not port_allowed(p)]
                count(disallowed) > 0
                msg := sprintf("NetworkPolicy %s/%s: spec.ingress[%d].ports[%d] opens the inclusive range %v-%v, which includes %d port(s) that are not allowed (first: %v). Allowed ports are %v.",
                  [ns, nm, i, j, port.port, port.endPort, count(disallowed), disallowed[0], allowed_ports])
              }

              # 5. Named ports resolve at the Pod, not here. Fail closed.
              violation contains {"msg": msg} if {
                some i, j
                port := input.review.object.spec.ingress[i].ports[j]
                is_string(port.port)
                msg := sprintf("NetworkPolicy %s/%s: spec.ingress[%d].ports[%d] uses the named port %q. Named ports resolve to a number at the Pod and cannot be checked here; use a numeric port from %v.",
                  [ns, nm, i, j, port.port, allowed_ports])
              }

Which workloads this applies to is now expressed in the constraint's match.labelSelector, where it belongs, rather than hardcoded in the policy.

Two honest caveats. This constrains the NetworkPolicy object, not the traffic — if the cluster's CNI does not enforce NetworkPolicy, this enforces the formatting of a document that has no effect. Create a deny-all policy in a test namespace and confirm the connection actually fails before investing here. And this covers spec.ingress only. The identical set of holes exists in spec.egress; the rules must be duplicated for it, and are not, because the article shows the pattern rather than a complete product.

The rejection message

A developer running kubectl apply against the 2021 NetworkPolicy constraint saw:

Historical — the rejection produced by the 2021 constraint. Reproduced to make a point about message design.

Error from server: admission webhook "validation.gatekeeper.sh" denied the request:
[deny-ingress-ports] Applying Ingress rule for the provided port is not allowed

That message names no port, no policy intent, no allowed values, no owner, and no next step. The developer's only path forward is to find the constraint, read the Rego, and infer the rule — or, far more likely, to file a ticket asking for an exemption. A denied deployment with an unhelpful message is a failed control, because the next thing that happens is an exemption request, and exemptions granted under delivery pressure do not get reviewed.

The rewrite produces:

Verified — the message emitted by the template above under gator test, with the constraint-name prefix Gatekeeper adds at admission.

Error from server: admission webhook "validation.gatekeeper.sh" denied the request:
[allowed-ingress-ports] NetworkPolicy payments/api-ingress:
spec.ingress[0].ports[1] allows port 8080; allowed ports are [443, 8443].

The mechanics are unremarkable: sprintf in Rego, messageExpression in CEL. That is the point. Nothing technical prevents a good message; what prevents it is that nobody owns the rejection as a user-facing surface. Treat it as one. Include the resource, the field path, the offending value, the permitted set, and an owner, plus a runbook URL if the policy has any nuance. Leave the constraint name out — Gatekeeper adds it, and reaching for input.constraint can silently disable the rule.

A referential constraint and its cache dependency

k8suniqueingresshost v1.0.4 is the canonical referential example: it cannot decide whether an Ingress host is unique without seeing every other Ingress. It carries a metadata.gatekeeper.sh/requires-sync-data annotation declaring that dependency, and that annotation does not cause the sync to happen.

Illustrative — the SyncSet that satisfies the template's declared data requirement. Adjust the GVK list to match your template's annotation exactly.

apiVersion: syncset.gatekeeper.sh/v1alpha1
kind: SyncSet
metadata:
  name: ingress-sync
spec:
  gvks:
    - group: "networking.k8s.io"
      version: "v1"
      kind: "Ingress"

gator sync test --filename=policies/ --filename=syncsets/ checks that the two agree. One flag matters before you trust it: --omit-gvk-manifest assumes all GVKs are supported, and upstream warns that "if this assumption is not true, then the given config and templates may cause caching errors or incorrect evaluation on the cluster despite passing this command." Supply a real GVK manifest from the target cluster if you want the check to mean anything.

Rollout: dryrun to warn to deny

Gatekeeper supports three enforcement actions: deny (the default), dryrun, and warn.

Illustrative — constraint promotion state machine. Every stage has a rollback edge; the gate between stages is audit evidence, not elapsed time alone.

Figure 3: flowchart diagram showing Constraint written, gator verify green in CI, enforcementAction: dryrun, Read audit violations, Violations triaged to zero, exempted, or accepted?, enforcementAction: warn, New violation classes appearing?, enforcementAction: deny

One caution about the first edge. A green gator verify is a necessary gate, not a sufficient one, and the two silent-drop failure modes above are exactly where green means nothing: a rule that never fires passes every positive test. Every constraint needs at least one test that asserts a violation. A suite of only "this should be allowed" cases passes against a policy that does nothing.

What to read at each stage

dryrun evaluates the constraint and records violations without affecting the request. The trap: --log-denies is false by default, and that flag governs logging for "all deny, dryrun and warn failures". Out of the box, a dryrun rollout produces no admission log line at all. Your only signal is constraint status, which is capped and reports only the most recent audit run. Turn logDenies on for the duration of a rollout, or export violations, or accept that you are flying on a lossy snapshot.

warn returns the message to the client as a warning while admitting the request. This is where message quality gets tested for real, and where you find out that half your applies go through a CI runner that discards warnings. Check that before relying on it.

deny rejects. Promote only when audit has been clean, or knowingly exempted, across a period that covers your slowest deployment cadence — including monthly batch jobs, quarterly certificate rotations, and whatever the disaster-recovery drill creates.

Scoped enforcement, and the audit trap

enforcementAction: scoped with scopedEnforcementActions sets different actions at different enforcement points. The obvious application is "deny in CI, warn at admission" — the policy blocks the merge but does not block the emergency deploy.

Illustrative — scoped enforcement. Note the explicit inclusion of the audit enforcement point; omitting it silently disables audit reporting for this constraint.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedIngressPorts
metadata:
  name: allowed-ingress-ports
spec:
  enforcementAction: scoped
  scopedEnforcementActions:
    - action: deny
      enforcementPoints:
        - name: "gator.gatekeeper.sh"
    - action: warn
      enforcementPoints:
        - name: "validation.gatekeeper.sh"
    - action: dryrun
      enforcementPoints:
        - name: "audit.gatekeeper.sh"
  match:
    kinds:
      - apiGroups: ["networking.k8s.io"]
        kinds: ["NetworkPolicy"]
  parameters:
    allowedPorts: [443, 8443]

The trap is documented and easy to miss: "the audit enforcement point is not included unless explicitly added to scopedEnforcementActions.enforcementPoints or if … set to "*"." A team that adopts scoped actions for "deny in CI, warn in cluster" and names only those two points has silently turned off audit for that constraint. The violations stop appearing, which looks exactly like success.

The Gatekeeper-to-VAP action mapping, if you are generating VAPs: deny becomes Deny, warn becomes Warn, dryrun becomes Audit.

CloudSecOps recommendation (judgement, not measurement): two full deployment cycles in dryrun with audit reviewed at the end of each, then one cycle in warn, then deny. For a team deploying daily that is roughly two weeks. For a team deploying monthly it is a quarter, and shortening it because a quarter feels long is how you discover that the monthly batch job mounts a host path.

Testing in CI with gator

gator is beta as of v3.11 and is the reason "we could not test the policy" is no longer a defensible position.

CommandWhat it doesWhere it belongs
gator verifyRuns Suite/Test/Case fixtures in suite.yaml filesPre-merge, on every policy change
gator testEvaluates arbitrary objects against templates and constraintsPre-merge, on application manifests
gator expandRenders workload controllers into the Pods they would createBefore gator test, for Deployments
gator sync testChecks requires-sync-data against SyncSets and ConfigPre-merge, when referential policies change
gator benchLatency percentiles, throughput, memory profilingNightly, with baseline comparison
gator policyInstalls upstream library policies (alpha)Manual, tracked by gatekeeper.sh/managed-by: gator

Three behaviours confirmed against the v3.23.0 binary while writing this article:

Exit codes. gator test exits 1 on violation of a constraint with deny or an empty enforcementAction, and 0 for dryrun — verified by running the same violating object against the same template with only the action changed. A dryrun constraint will not fail your pipeline, which is correct during rollout and a footgun if you forget it is still in dryrun six months later. Alert on it rather than relying on the exit code.

--enable-k8s-native-validation defaults to true, confirmed in gator test --help, so local tests evaluate the CEL engine with the same precedence as the cluster. That is how you catch the dual-engine no-op described earlier.

gator reads stdin by default. Without --filename=- it still blocks on standard input if stdin is an open pipe. Interactively this looks like a hang; in CI it looks like a job that times out for no visible reason. Redirect from /dev/null in automation.

Production-ready — the gator verify suite for the host-path template above. This exact suite passes against the exact template shown; gator verify ./... is how the upstream library tests itself.

apiVersion: test.gatekeeper.sh/v1alpha1
kind: Suite
tests:
  - name: allowed-host-paths
    template: template.yaml
    constraint: constraint.yaml
    cases:
      - name: allows-a-permitted-prefix
        object: samples/pod-allowed.yaml
        assertions:
          - violations: no
      - name: rejects-a-sibling-prefix
        object: samples/pod-fool.yaml
        assertions:
          - violations: yes
            message: "not under an allowed prefix"
      - name: rejects-writable-mount-of-allowed-path
        object: samples/pod-allowed-rw.yaml
        assertions:
          - violations: yes
            message: "must be mounted readOnly"

The second case earns its keep: pod-fool.yaml mounts /fool when /foo is allowed, which is how you stop a future refactor replacing segment-aware matching with startswith.

Write the negative cases first. Every bypass in the NetworkPolicy section became a case in that policy's suite, and each was written because running the policy showed it was needed — not because anyone reasoned their way to it.

For workload controllers, chain expansion into the test:

Illustrative — expanding Deployments into their generated Pods before evaluating constraints. gator expand fails if a referenced Namespace object is not supplied, so include namespace manifests in the input.

gator expand --filename=manifests/ | \
  gator test --filename=- --filename=policies/

Observability and ownership

Gatekeeper exports Prometheus metrics on port 8888 at /metrics by default, and supports OTLP via --otlp-endpoint.

Metric and conditionWhat it meansSuggested severity
gatekeeper_constraints with status="error" above 0A template failed to compile or a CRD is missing; that constraint enforces nothingPage
gatekeeper_validation_request_count flat while API traffic continuesThe webhook is not being called; enforcement has stoppedPage
gatekeeper_validation_request_duration_seconds p99 approaching 3sTimeouts imminent; fail-open will begin admitting silentlyPage
gatekeeper_audit_last_run_end_time stale beyond 3× --audit-intervalAudit is stuck or overlapping; violation reporting is unreliableTicket
gatekeeper_sync_last_run_time stale after a restartReferential constraints may be evaluating against an incomplete inventoryTicket
gatekeeper_violations for enforcement_action="dryrun" not trending downA rollout has stalled; the constraint will sit in dryrun foreverTicket
gatekeeper_mutator_conflicting_count above 0Two mutators contend for the same field; final object is unpredictableTicket

gatekeeper_constraints is tagged by enforcement_action and status (active or error), making two useful questions one-liners: how many constraints are broken right now, and how many are still in dryrun ninety days after someone promised to promote them.

Note what no metric can tell you. A constraint whose rule silently never fires reports status="active", zero violations, and normal request counts — indistinguishable from a compliant cluster. The only defence is a negative test in CI.

Per-constraint statistics are available via --log-stats-admission and --log-stats-audit, with the upstream caveat that "the additional log volume from enabling the stats logging can be quite high." Turn them on to diagnose latency; turn them off afterwards. And do not forget the API server's own admission webhook metrics: Gatekeeper cannot report that it was never called. Only the API server can.

Ownership

Three roles, and conflating them is how constraints end up unowned:

  • The platform team owns Gatekeeper as a component: version, availability, exemption flags, the break-glass runbook, and the metrics above. If the webhook is down, this is their page.
  • The policy owner owns an individual constraint: its parameters, its rollout stage, its rejection message, and the audit backlog it generates. Encode this in a constraint annotation, because annotations appear in audit log events as constraint_annotations and become a routing key.
  • The approver owns exceptions. This should not be the same person as the policy owner, and it should not be whoever is on call.

Exceptions

Exceptions are a product surface. Design them or they will be designed for you, badly, at 4pm on a Friday. Gatekeeper offers four mechanisms with wildly different blast radii, and upstream offers no guidance on choosing. CloudSecOps' ranking, strongest to weakest:

1. Narrow the constraint's match. A labelSelector, an excludedNamespaces entry on the constraint, or a name filter. The exception is visible in the manifest, versioned in Git, reviewable in a pull request, and scoped to exactly the resources you named. Everything else still enforces. This should be the default answer.

2. Config.spec.match.excludedNamespaces. Broader — it disables evaluation for a namespace across all constraints — but the webhook is still called and the exclusion is one reviewable object. Use for namespaces, not workloads.

3. Drop the constraint to warn. Keeps the signal, loses the enforcement, applies globally to that constraint. Acceptable as a time-boxed rollback during an incident; unacceptable as a permanent state, and the metric above exists to catch it becoming one.

4. Namespace exemption via admission.gatekeeper.sh/ignore. Last resort. Disables all Gatekeeper policy in the namespace, invisible unless you inspect namespace labels, and anyone who can label a namespace can grant it to themselves. Reserve it for availability-critical namespaces decided once at install time.

Whatever mechanism you use, three properties are non-negotiable: an expiry date, a named approver who is not the requester, and a recorded reason specific enough to re-evaluate later. "Needed for the migration" is not a reason. "Needs /var/run/docker.sock until the build agents move to Kaniko, tracked in PLAT-4471" is.

Gatekeeper, ValidatingAdmissionPolicy, or Kyverno

Illustrative — routing a proposed guardrail to the control that should own it. The first two branches route most policies away from admission control entirely.

Figure 4: flowchart diagram showing You want to stop X from happening, Is X really an identity or permission question?, Must it hold at runtime, not only at creation?, Does the decision need state beyond the submitted object?, Do you need the same rule in CI and in cluster?, Do you need to find exis

The first branch is not rhetorical. The 2021 series' own "Restrict NetworkPolicy Management to Specific Users (Part 1)" contains no Gatekeeper policy at all — it is a Role and RoleBinding walkthrough. That was the correct answer to that question, and it means the original series already made this article's central argument implicitly. Some guardrails belong in RBAC, and writing them as constraints makes them weaker, because RBAC cannot fail open.

DimensionValidatingAdmissionPolicyGatekeeperKyverno
LanguageCELRego and CELYAML DSL and CEL
Where it runsIn the API server processSelf-hosted webhookSelf-hosted webhook
Failure modeNo webhook to failFail-open by defaultFail-closed by default
Referential stateLimited (param resources)Yes, via SyncSet inventoryYes, via API calls
Audit of existing resourcesNoYes, audit controllerYes, background scans
Shift-left CLINo first-party equivalentgator (beta)kyverno CLI
MutationMutatingAdmissionPolicy, stable 1.36Stable since v3.10Yes, mature
Operational surfaceZero extra componentsTwo deployments, three webhooks, CRDsMultiple controllers, dedicated namespace

Kyverno v1.18's ValidatingPolicy type is built to extend and generate ValidatingAdmissionPolicy rather than compete with it — the direction Gatekeeper took with VAP generation. Both are converging on "author once, run in-tree where possible, fall back to the webhook where not". Compare them on operational model, not feature checkboxes; the checkboxes move quarterly.

CloudSecOps position: for a greenfield cluster in 2026, a simple single-object validation should start as a ValidatingAdmissionPolicy. A policy running in the API server process has no webhook to be unavailable, no certificate to rotate, no memory limit to exceed and no 3-second timeout. Gatekeeper earns its place on five things: referential policies, external data, audit of pre-existing resources, the same artifact evaluated in CI via gator, and one constraint running at all four enforcement points. If your policy set needs none of those, you may be operating a webhook for nothing.

That position is sensitive to architecture, and three cases invert it:

  • Managed distributions. On AKS with Azure Policy, or GKE with Policy Controller, Gatekeeper is already installed, the version is chosen for you, and the supported path is the vendor's constraint catalogue. Installing upstream Gatekeeper alongside is a supportability problem. The decision tree above applies to what you add, not to whether you run Gatekeeper at all.
  • Compliance evidence. If an auditor needs a periodic report of non-compliant resources that already exist, VAP cannot produce it and Gatekeeper's audit controller can. That single requirement is often decisive regardless of policy complexity.
  • Fleets. Across many clusters, the operational cost of Gatekeeper is paid once in tooling and repeated per cluster in resources; the cost of VAP is near zero per cluster but the policies are harder to test uniformly. Small fleet of large clusters favours Gatekeeper; large fleet of small clusters favours VAP.

Mutation, briefly

Gatekeeper mutation has been stable since v3.10, with three mutator kinds at v1 (Assign, AssignMetadata, ModifySet) and one at v1alpha1 (AssignImage).

Most teams should not start here. Mutating webhooks run in sequence, before validation, and the shipped mutating timeout is 1 second — three times tighter than validation. Every mutator is serial latency on every matching request.

Kubernetes' guidance is to fail open on mutation and validate the final state: assume your mutator might not have run, and write a validating constraint that catches the case where the field you meant to set is absent. A mutation that silently does not happen is worse than none, because downstream systems assume the field is there.

Mutators must be idempotent. reinvocationPolicy defaults to Never and Gatekeeper keeps that default, but other webhooks may mutate after you, and a reinvocation-enabled configuration will call you again on your own output. Watch gatekeeper_mutator_conflicting_count; above zero means two mutators contend for one field and the result depends on ordering you do not control.

Minimum viable action list

Ordered. Each item is checkable.

  1. Record your versions. Gatekeeper app version, embedded OPA version, Kubernetes minor, and whether you are on upstream or a vendor redistribution. Put it in the runbook.
  2. Confirm your failurePolicy and know what it means. Default is Ignore on both policy webhooks and Fail on check-ignore-label. Decide deliberately; do not inherit.
  3. Alert on webhook unavailability before you write a second constraint. gatekeeper_validation_request_count flat and gatekeeper_constraints with status="error" above zero are the two that matter most.
  4. Exempt what must work during recovery. kube-system at minimum, plus kube-node-lease, your CNI and CSI namespaces, and any service-mesh control plane. Use --exempt-namespace for these, not Config.
  5. Audit who can label namespaces. patch on namespaces is equivalent to policy bypass. Alert on any namespace acquiring admission.gatekeeper.sh/ignore.
  6. Write the break-glass runbook and test it, including suspending GitOps reconciliation before deleting the webhook configuration. Pair it with an audit-log detection on webhook-configuration writes.
  7. Put gator verify in CI and make it a required check. Include at least one test per constraint that asserts a violation — a suite of only-allow cases passes against a policy that does nothing.
  8. Bind every optional parameter through object.get with a default, and keep input.constraint out of your messages. Both prevent silent rule drops.
  9. Start every constraint in dryrun. No exceptions, including for constraints you are certain about.
  10. Turn on logDenies for the duration of a rollout, or wire up violation export. The default gives you no admission log line for a dryrun violation.
  11. Read audit before promoting. Raise constraintViolationsLimit toward 500 first, or you will be reading a truncated list.
  12. Write the rejection message before the rule. Resource, field path, offending value, permitted set, owner, runbook.
  13. Annotate every constraint with an owner. It shows up in audit log events and becomes your routing key.
  14. Review what should not be in Gatekeeper at all. Run each existing constraint through the decision tree. Anything that is an identity question moves to RBAC; anything expressible in CEL against a single object with no CI requirement moves to ValidatingAdmissionPolicy.
  15. Set a review date on every exception, and an owner who is not the requester.

The 2021 series

This article consolidates a ten-part series originally published here in 2021. Those posts are no longer served; the archived originals are linked in the references below. The typo in one legacy slug (resrict-networkmanagement-to-specific-u) is reproduced there exactly, because that is the URL that carries the inbound links.

References

Gatekeeper project documentation and artifacts

Kubernetes documentation

Open Policy Agent

Kyverno (comparison only)

Recovered CloudSecOps originals (Internet Archive)

Validity and revision

Verification date: 2026-08-06. Every version number, default value and quoted behaviour above was checked against the primary source on that date, using the shipped Gatekeeper manifest at tag v3.23.0, the project Helm repository index, the Kubernetes API type definitions, and current Kubernetes and OPA documentation.

What was executed. The CloudSecOps policies were compiled with OPA 1.19.0 (opa check --v1-compatible) and run through gator verify using the gator CLI built from upstream source at tag v3.23.0. Fifteen cases pass across three suites. Gatekeeper v3.23.0 embeds OPA v1.17.1, so the standalone compiler is slightly ahead of the one in the cluster; Rego v1 syntax is stable across that gap, but the difference is stated rather than glossed. The gator results came from the matching version.

Amended after that run. Three policy corrections post-date the recorded gator results and have not been through it: the empty-ports case in the NetworkPolicy template (ports: [] is present-but-empty and was not caught by an undefined check), scoping the RBAC name exemption to ClusterRole, and treating prefix wildcards such as /* and */exec as wildcards. Re-run the suite with cases for each before relying on these in a cluster. The counts above describe the earlier revision, and are left unchanged rather than restated from an unrun suite.

What was not executed. Nothing ran on a live cluster: no webhook invoked, no audit controller, no data cache, no latency measured. Claims about admission-time and audit-time behaviour rest on upstream documentation and the shipped manifest, not observation.

What is version-dependent and will move first:

  • Gatekeeper version. v3.23.0 is stable; v3.24.0-beta.0 was published 2026-07-13. Minor releases land roughly quarterly.
  • --sync-vap-enforcement-scope is documented as deprecated and slated for removal in v3.24.
  • Kubernetes supported minors. 1.34 reaches end of life on 2026-10-27. Gatekeeper's minimum supported version tracks the Kubernetes supported-versions policy — the three most recent minors.
  • MutatingAdmissionPolicy is stable and default-on as of 1.36, but ecosystem adoption is early. Treat the mutation comparison as a statement about capability, not about maturity of tooling.
  • Rego v0 as Gatekeeper's default is a compatibility position, not a destination. Expect the default to move.
  • input.constraint availability by enforcement point. The behaviour documented here is v3.23.0's. It could reasonably change, in either direction, and the guarded pattern is safe under both.
  • Upstream library policy versions, pinned above by their metadata.gatekeeper.sh/version values. Community-maintained and versioned per policy; re-check before adopting.
  • Kyverno v1.18.2 is current stable with v1.19.0-rc.1 published 2026-08-03.

Known evidence limits. The failure-mode analysis rests on upstream documentation, the shipped manifest, and the execution above, without third-party incident write-ups; the evidence base is narrow even though every claim is sourced. The cold-cache row is design reasoning, not documented or tested behaviour, and is marked as such. The kubernetes.io/bootstrapping: rbac-defaults label suggested for RBAC exemptions was not verified against a running cluster. The NetworkPolicy template covers spec.ingress only.

Recommended review date: 2027-02-06, or immediately on Gatekeeper v3.24 reaching general availability, whichever is sooner.

  • kubernetes
  • opa
  • gatekeeper
  • admission-control
  • policy-as-code
  • rego
  • cel
  • devsecops

The service behind this work

Cloud security assessment

A senior review of how your AWS and Google Cloud estate is actually built and governed — scored against a control checklist you keep, and turned into a remediation roadmap your board can read and your engineers can execute.