Skip to content

linkerd install/upgrade --crds silently overwrites Helm-managed CRDs and irreversibly bumps storage versions, breaking subsequent helm upgrade #15471

Description

@jaswanth9522

What is the issue?

We use Linkerd Stable version 2.14.5 . We use linkerd crds as one of the Umbrella Chart in one of our base Chart and when we are trying to update the base helm chart there is no change in Linkerd Crds or Control plane helm charts but somehow we are getting this issue
cannot patch "httproutes.gateway.networking.k8s.io" with kind CustomResourceDefinition: CustomResourceDefinition.apiextensions.k8s.io "httproutes.gateway.networking.k8s.io" is invalid: status.storedVersions[1]: Invalid value: "v1": must appear in spec.versions && cannot patch "servers.policy.linkerd.io" with kind CustomResourceDefinition: CustomResourceDefinition.apiextensions.k8s.io "servers.policy.linkerd.io" is invalid: status.storedVersions[1]: Invalid value: "v1beta3": must appear in spec.versions but we didnt change the underlying crds version when this happen we happened to see that that crds were updated by {"linkerd.io/created-by":"linkerd/cli edge-26.1.2"} which we never did
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","metadata":{"annotations":{"linkerd.io/created-by":"linkerd/cli edge-26.1.2"},"labels":{"helm.sh/chart":"linkerd-crds-0.0.0-undefined","linkerd.io/control-plane-ns":"linkerd"},"name":"servers.policy.linkerd.io"},"spec":{"group":"policy.linkerd.io","names":{"kind":"Server","plural":"servers","shortNames":["srv"],"singular":"server"},"scope":"Namespaced","versions":[{"deprecated":true,"deprecationWarning":"policy.linkerd.io/v1alpha1 Server is deprecated; use policy.linkerd.io/v1beta1 Server","name":"v1alpha1","schema":{"openAPIV3Schema":{"properties":{"spec":{"properties":{"podSelector":{"description":"Selects pods in the same namespace.","oneOf":[{"required":["matchExpressions"]},{"required":["matchLabels"]}],"properties":{"matchExpressions":{"items":{"properties":{"key":{"type":"string"},"operator":{"enum":["In","NotIn","Exists","DoesNotExist"],"type":"string"},"values":{"items":{"type":"string"},"type":"array"}},"required":["key","operator"],"type":"object"},"type":"array"},"matchLabels":{"type":"object","x-kubernetes-preserve-unknown-fields":true}},"type":"object"},"port":{"description":"Aalso we didnt have linkerd cli installed as well.

How can it be reproduced?

Not able to reproduce this continuously and this is very much random.

Logs, error output, etc

`cannot patch "httproutes.gateway.networking.k8s.io" with kind CustomResourceDefinition: CustomResourceDefinition.apiextensions.k8s.io "httproutes.gateway.networking.k8s.io" is invalid: status.storedVersions[1]: Invalid value: "v1": must appear in spec.versions && cannot patch "servers.policy.linkerd.io" with kind CustomResourceDefinition: CustomResourceDefinition.apiextensions.k8s.io "servers.policy.linkerd.io" is invalid: status.storedVersions[1]: Invalid value: "v1beta3": must appear in spec.versions

output of linkerd check -o short

N/A

Environment

Kubernetes version: 1.32.4

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

None

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions