Everything-is-an-app: migrate existing clusters (offline settings + runtime diff-extraction)

[j-zimnowoda]

Parent

#3515

Design update (supersedes original §3/§8): offline settings migration populates AplCluster.spec.apps enable flags in env/settings/cluster.yaml (not env/settings/apps.yaml) via leaf-path values-changes.yaml relocations. See #3515 comment and ADR adr/2026-08-21-app-enable-flags-in-aplcluster.md.

What to build

The final step: migrate existing clusters onto the new layout without losing user configuration, delivered as two coordinated tracks.

End-to-end behavior:

• Offline settings migration — the existing otomi migrate / values-changes.yaml pipeline (kept, not retired) moves platform settings files into their new locations, including populating AplCluster.spec.apps enable flags in env/settings/cluster.yaml. Enable flags migrate as leaf-path relocations (apps.<name>.enabled: cluster.apps.<name>.enabled, plus nested gating exceptions such as apps.istio.egressGateway.enabled: cluster.apps.istio.egressGateway.enabled), so only enablement leaves move — the rest of apps.<name> flows to the runtime app-values track below.
• Runtime app-values migration — a one-time, cluster-aware operator upgrade step reconstructs each app's values.custom.yaml from the running cluster:
  1. Read the existing Application's effective values from the cluster (inline spec.source.helm.values, fallback helm get values <release>).
  2. Render the app's new values.managed.yaml.
  3. values.custom.yaml = deepDiff(existingEffective, managed) — a key lands in custom iff it is absent from or differs from managed (minimal custom.yaml).
  4. Write custom + managed, commit, and let the multi-source Application (same <namespace>-<release> name) supersede the inline one as an update, not a replace.
• Because the cluster already holds vendor-shaped effective values, this auto-handles curated→vendor translation and multi-release splitting with no hand-authored per-app mapping.

Must run after all cutover batches so the new layout exists for every app.

Acceptance criteria

• otomi migrate moves platform settings and populates AplCluster.spec.apps enable flags in env/settings/cluster.yaml (leaf-path relocations) on an existing values repo.
• The runtime step reconstructs a minimal values.custom.yaml per app via deepDiff(existing effective values, managed) and commits custom + managed.
• The updated multi-source Application supersedes the inline one in place (same <ns>-<release> name) with no drift / no destroy-recreate.
• A customized app on an existing cluster retains its customizations after migration.
• Unit tests cover the diff rule (absent-or-differing → custom) with fixtures, no real git/K8s.

Blocked by

• Everything-is-an-app cutover: Ingress/DNS/TLS group (istio, kubernetes-gateways, external-dns) #3575
• Everything-is-an-app cutover: SSO/OIDC group (keycloak, oauth2-proxy, argocd, otomi-console) #3576
• Everything-is-an-app cutover: Storage & Database group (harbor, gitea, loki, kubeflow-pipelines) #3577
• Everything-is-an-app cutover: Monitoring group (prometheus, grafana, promtail, alertmanager, otel) #3578
• Everything-is-an-app cutover: standalone apps + delete env/apps + secrets audit #3579