Administrators cannot repair (or delete) namespaces with an inconsistent tenant reference (label without Tenant ownerReference)

[antoinemichea]

Bug description

A namespace that carries the tenant label (capsule.clastix.io/tenant) without the matching Tenant ownerReference can neither be updated nor deleted by anybody — including identities listed in CapsuleConfiguration.spec.administrators. The inconsistency can therefore never be repaired through the API.

Such half-assigned namespaces are not hypothetical: they get created whenever a namespace is admitted while the Capsule webhooks are unavailable (operator outage/crashloop, failurePolicy: Ignore window, or objects that predate the 0.13 consistency model). We hit this in production with a namespace created during a controller outage: it was invisible to quota/NetworkPolicy replication AND impossible to fix.

How to reproduce

1. Produce an inconsistent namespace (simulates a webhook outage leftover), e.g. scale the capsule operator to 0 with failurePolicy: Ignore, or as an administrator strip the ownerReference from an assigned namespace.
2. Add the repairing identity to CapsuleConfiguration.spec.administrators (e.g. {kind: Group, name: system:masters}).
3. Try to repair it as that administrator:

kubectl patch ns broken-ns --type=merge -p '{"metadata":{"ownerReferences":[{"apiVersion":"capsule.clastix.io/v1beta2","kind":"Tenant","name":"my-tenant","uid":"<tenant-uid>","controller":true}]}}'

Actual result (any update, even the one that would restore consistency — and kubectl delete as well):

Error from server: admission webhook "namespaces.validating.projectcapsule.dev" denied the request: namespace has tenant label "my-tenant" but no Tenant ownerReference

Expected: administrators should be able to repair (or delete) such namespaces — that's precisely what administrators is for, and the mutating guard handler already fully bypasses admins.

Root cause

In internal/webhook/namespace/validation/handler.go, OnUpdate calls tenant.ResolveNamespaceTenant() on the old and new objects unconditionally and turns any error into a denial before the user.IsAdmin() bypass can apply (ResolveNamespaceTenant errors on label != "" && len(refs) == 0). OnDelete has the same pattern, so an inconsistent namespace cannot even be removed. Verified on v0.13.9 and current main.

Related: #1941 addressed admin updates of tenant-less namespaces; this issue is about inconsistent ones (label without ownerReference, and the other mismatch states ResolveNamespaceTenant rejects).

Workaround

Temporarily exclude the affected namespace from the namespaces validating hook via webhooks.hooks.namespaces.objectSelector (chart values), patch, then revert — clumsy for something administrators should allow.

Proposed fix

Treat resolution errors as non-fatal for administrators only in OnUpdate/OnDelete: the inconsistent object is handled as tenant-less for them, which permits the repair (restoring the ownerReference or removing the stale label) as well as deletion. OnCreate is left strict (the mutating assignment webhook guarantees consistency on that path) and nothing changes for non-administrators.

PR incoming with the fix and an e2e case in e2e/config_administrators_test.go.

Environment

• Capsule v0.13.9 (also present on main)
• Kubernetes v1.35, cert-manager-managed webhook certificates