CRD source ignores `--crd-source-apiversion` / `--crd-source-kind` in v0.21.0

[IceHamster]

What happened:

external-dns v0.21.0 regressed the crd source and appears to ignore --crd-source-apiversion / --crd-source-kind for custom CRDs implementing the DNSEndpoint contract.

In our cluster, this worked with Helm chart 1.20.0 (app v0.20.0) and stopped working with Helm chart 1.21.1 (app v0.21.0).

The parsed config in both versions still shows:

Sources:[service ingress crd]

and in the broken version also shows the custom CRD settings:

CRDSourceAPIVersion:externaldns.nginx.org/v1
CRDSourceKind:DNSEndpoint

so this does not look like a --source parsing issue. The regression appears to be in the CRD source implementation after flag parsing.

What you expected to happen:

The crd source should continue to honor:

• --crd-source-apiversion
• --crd-source-kind

for custom CRDs implementing the DNSEndpoint contract, as documented in:

• docs/sources/crd.md
• docs/flags.md

and as observed in v0.20.0.

How to reproduce it (as minimally and precisely as possible):

1. Run external-dns with a custom DNSEndpoint-style CRD, for example:

--source=service
--source=ingress
--source=crd
--crd-source-apiversion=externaldns.nginx.org/v1
--crd-source-kind=DNSEndpoint
--provider=azure
--domain-filter=dev.domain.com
--registry=txt
--txt-owner-id=<owner-id>

2. Create custom resources under that apiVersion/kind which implement the DNSEndpoint contract and previously produced DNS records on v0.20.0.
3. Compare behavior between:
   • Helm chart 1.20.0 / app v0.20.0
   • Helm chart 1.21.1 / app v0.21.0

Observed result:

• v0.20.0: CRD-derived records are reconciled and written.
• v0.21.0: config still includes Sources:[service ingress crd], but the CRD-derived records are no longer processed.

Repository-level investigation points to ea4d2d16 / PR #6312 (refactor(source/crd): migrate CRD source to controller-runtime cache) as the regression window. The old dynamic CRD path that used the configured apiVersion/kind was replaced with a typed implementation over the built-in externaldns.k8s.io/v1alpha1 DNSEndpoint.

Anything else we need to know?:

• This looks unintended rather than intentional:

  • the change was introduced as a refactor, not a documented breaking change
  • PR #6312 describes an internal migration to controller-runtime cache and does not mention removing support for custom CRD apiVersion/kind
  • the PR checklist left end-user documentation updates unchecked
  • the CLI still exposes --crd-source-apiversion and --crd-source-kind
  • docs/sources/crd.md still says the CRD source works by specifying apiVersion and kind
  • the chart CRD still describes DNSEndpoint as a contract that a user-specified CRD may implement

• I also checked the newer unstructured source. It does not look like a documented replacement for this behavior:

  • it is template-driven (--source=unstructured, --unstructured-resource=..., --fqdn-template, --target-template)
  • it is not a drop-in replacement for the old DNSEndpoint contract
  • no migration away from custom crd apiVersion/kind support is documented

• A local fix branch restores the old behavior by:

  • keeping the typed/cache-backed path for the built-in externaldns.k8s.io/v1alpha1 DNSEndpoint
  • falling back to a dynamic REST/discovery-backed path when a custom CRD apiVersion/kind is configured

Environment:

• External-DNS version (use external-dns --version):
  • broken: ExternalDNS/v20260406-v0.21.0
  • working: ExternalDNS/v20251114-v0.20.0
• DNS provider:
  • Azure
• Others:
  • working chart: external-dns-helm-chart-1.20.0
  • broken chart: external-dns-helm-chart-1.21.1
  • custom CRD settings in use:
    • --crd-source-apiversion=externaldns.nginx.org/v1
    • --crd-source-kind=DNSEndpoint

Checklist

• I have searched existing issues and tried to find a fix myself
• I am using the latest release,
  or have checked the staging image to confirm the bug is still reproducible
• I have provided the actual process flags (not Helm values)
• I have provided kubectl get <resource> -o yaml output including status
• I have provided full external-dns debug logs
• I have described what DNS records exist and what I expected

[ivankatliarchuk]

Not sure if this is a bug. The flags may have been informational, not functional for custom GVKs. --crd-source-apiversion and --crd-source-kind have always defaulted to externaldns.k8s.io/v1alpha1 / DNSEndpoint. Every example in docs/sources/crd.md and docs/flags.md shows only those default values. There is no documentation, example, or guidance for pointing them at a third-party CRD like externaldns.nginx.org/v1.

source/crd_test.go has zero coverage for a non-default apiVersion/kind. If this use case had been intentionally supported, it would have had tests. The absence of any test for the custom GVK path means there was no regression safety net - and arguably no formal contract with users/api.

The right question is: should custom CRD GVK support be an officially supported feature? If yes, it needs documentation with a real custom GVK example, unit tests covering the non-default path, and a deliberate implementation. If no, the flags should either be deprecated or their description clarified to reflect that they only confirm the built-in defaults.

[ivankatliarchuk]

What is the your use case for custom CRD? On the same note unstructured source does work with any CRD, but not a replacement, as this was not a known feature.

[IceHamster]

Our use case is the F5 NGINX Ingress Controller ExternalDNS integration.

We are using the dnsendpoints.externaldns.nginx.org CRD from F5 NGINX:

https://github.com/nginx/kubernetes-ingress/blob/main/config/crd/bases/externaldns.nginx.org_dnsendpoints.yaml
https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/#virtualserverexternaldns

The resources are:

apiVersion: externaldns.nginx.org/v1
kind: DNSEndpoint

This worked with chart 1.20.0 / app v0.20.0: ExternalDNS processed those DNSEndpoint resources and updated the DNS records.

After upgrading to chart 1.21.x / app v0.21.0, the same configuration no longer works with:

--source=service
--source=ingress
--source=crd
--crd-source-apiversion=externaldns.nginx.org/v1
--crd-source-kind=DNSEndpoint

ExternalDNS still accepts the flags, but the CRD records are no longer reconciled.

[ivankatliarchuk]

Interesting, I'll ask around to see if anyone is aware of this feature - a 3rd party product documenting something for this service without the maintainers knowing it 😄.

F5 NGINX Controller and ExternalDNS are two separate products, so I'm not sure where or how this should be documented on our end. This level of CRD coupling probably shouldn't exist in the first place, as it works against both projects. As far as I know, NGINX Controller is also EOL so how this should be handled is unknown as well.

Worth reaching out to whoever maintains/supports nginx controller and ask about potential solutions - migration docs or similar would be a good starting point.

[IceHamster]

ingress-nginx (the Kubernetes community project) is the one being archived. F5's NGINX Ingress Controller is a separate, actively maintained project with both an open-source and a commercial edition.

Regarding unstructured, I do not know if it can fully replace the CRD source. It may work for simple A / AAAA / CNAME cases, but I am unsure about TTL and other parts of the DNSEndpoint behavior.

On whether custom CRD GVK support was a real feature in v0.20.0:

• Docs say "user specified CRD" (docs/sources/crd.md#L7-L8) selected via --crd-source-apiversion / --crd-source-kind (docs/sources/crd.md?plain=1#L71-L75.
• Tests cover a non-default test.k8s.io/v1alpha1 GVK (source/crd_test.go#L246-L286), (types_test.go#L238-L239).
• Implementation used the supplied apiVersion/kind via discovery (source/crd.go#L66-L117.

Dropping the custom CRD GVK in PR #6312 without a deprecation notice or migration path looks like an unintentional regression.

[ivankatliarchuk]

Gotcha. We are going to add a depreciation in next release, as theyl releases are immutable for security reasons we could not patch current release.

[pdabelf5]

I just want to add, at the time the integration was done between NGINX Ingress Controller & ExternalDNS, specifying a CRD source with --crd-source-apiversion / --crd-source-kind was (as far as I can tell) the documented approach available for integration at the CRD level to the NGINX Ingress Controller team.

We can look at alternative approaches for integration if user specified CRD's are no longer an option, although maintaining compatibility for existing users would be preferable. Any assistance or suggestions you can provide would be appreciated to maintain compatibility.

[ivankatliarchuk]

Are we talking about the community ingress-nginx controller or the F5 NGINX Ingress Controller? The community one is EOL, so migrating away makes sense - and in that context, using a locked ExternalDNS version alongside it is perfectly fine as a transitional setup.

As for F5 NGINX support, there are a couple of approaches: either a dedicated source (similar to how [f5_virtualserver.go])https://github.com/kubernetes-sigs/external-dns/blob/master/source/f5_virtualserver.go) handles F5 VirtualServer resources), or extending the existing unstructured source with the missing capabilities. Either way, the key principle here is loose coupling between the projects - which means work will likely be needed on both sides.

One of the challenges we have - path to beta issue example #5243, and coupling this way to controllers will not let us move forward.