Skip to content

fix: move cluster pki and kubeconfig out of binary_dir - #3198

Merged
kubesphere-prow[bot] merged 1 commit into
kubesphere:mainfrom
redscholar:fix/move-pki-kubeconfig-to-workdir
Aug 19, 2026
Merged

fix: move cluster pki and kubeconfig out of binary_dir#3198
kubesphere-prow[bot] merged 1 commit into
kubesphere:mainfrom
redscholar:fix/move-pki-kubeconfig-to-workdir

Conversation

@redscholar

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind bug

What does this PR do

Moves the cluster-specific pki and kubeconfig artifacts from binary_dir to work_dir, so that a single binary_dir can be safely shared across multiple clusters.

Background / Motivation

binary_dir is meant to be a cluster-agnostic, shareable offline package (binaries + images). However, certs/init, kubernetes/init, certs/renew, image-registry and precheck currently write/read the generated pki and kubeconfig under binary_dir. These are cluster-specific artifacts.

When the same binary_dir is intentionally shared across clusters (to reuse the offline artifact package), {binary_dir}/pki and {binary_dir}/kubeconfig collide between clusters, corrupting the CA/kubeconfig of every cluster sharing that directory.

Implementation

Replace all {{ .binary_dir }}/pki and {{ .binary_dir }}/kubeconfig references with {{ .work_dir }}/pki and {{ .work_dir }}/kubeconfig. work_dir is already per-cluster, so the artifacts are naturally isolated while binary_dir stays shareable.

runtime was deliberately not used as the target: it stores per-run execution state (CRs / variable caches) under a deep, run-scoped path and may be cleaned, whereas pki/kubeconfig must persist across runs (e.g. gen_cert IfNotPresent reuse, certs/renew, join) and be referenced by independent playbooks.

Key Changes

File What changed
builtin/core/roles/certs/init/tasks/main.yaml All gen_cert out_key/out_cert + final chown now use {{ .work_dir }}/pki
builtin/core/roles/certs/renew/kubernetes/tasks/kube.yaml fetch/copy kubeconfig now {{ .work_dir }}/kubeconfig
builtin/core/roles/certs/renew/image-registry/* cert copy src → {{ .work_dir }}/pki
builtin/core/roles/kubernetes/init-kubernetes/tasks/main.yaml fetch admin.conf → {{ .work_dir }}/kubeconfig
builtin/core/roles/kubernetes/join-kubernetes/tasks/main.yaml copy kubeconfig src → {{ .work_dir }}/kubeconfig
builtin/core/roles/image-registry/{harbor,docker-registry}/tasks/install.yaml cert copy src → {{ .work_dir }}/pki
builtin/core/roles/precheck/image-registry/network/tasks/main.yaml local check.sh + add_hostvars cert paths → {{ .work_dir }}/pki
builtin/core/roles/defaults/defaults/main/02-image_registry.yaml ca_file/cert_file/key_file defaults → {{ .work_dir }}/pki
builtin/core/roles/defaults/defaults/main/03-kubernetes.yaml doc comment path → {{ .work_dir }}/pki
builtin/core/roles/defaults/defaults/main/04-etcd.yaml etcd ca_file/server_*/client_*{{ .work_dir }}/pki
docs/en/reference/config.md, docs/zh/reference/config.md example paths updated
pkg/const/workdir.go structure comment updated to show pki/ and kubeconfig under work_dir

Impact

  • Affected modules: builtin playbooks/roles (certs, kubernetes, image-registry, precheck), defaults, const, docs.
  • API changes: N/A (no Go API / CRD change).
  • Database / state changes: pki/kubeconfig now live under work_dir instead of {{ work_dir }}/kubekey; rollback = revert this commit.
  • Config changes: N/A — the affected paths are internal artifacts, not user-facing config keys. Documented example paths updated only.
  • Dependency changes: N/A.

Breaking Changes

None for the standard single-cluster flow. The change is a local-path-only template substitution: all {{ .work_dir }}/pki / {{ .work_dir }}/kubeconfig references are on the kk control host (gen_cert writes, copy/fetch sources/destinations, local precheck check.sh). The certificates are still copied to the same remote destinations (/etc/ssl/etcd/ssl, /opt/harbor/*/ssl, /opt/docker-registry/*/ssl), so install and cert-renew behavior is unchanged.

Only external automation that explicitly read {binary_dir}/pki or {binary_dir}/kubeconfig would need to look under work_dir instead.

Which issue(s) this PR fixes:

Fixes #

Testing

Verification performed

  • Code self-reviewed, no debug code or commented-out dead code
  • Unit tests pass (make test)
  • Integration / E2E tests pass (make test)
  • Manual verification on a live cluster

Steps to verify

  1. make build (or make kk) — embeds the updated builtin playbooks.
  2. kk create cluster with a shared binary_dir across two clusters — confirm work_dir/pki and work_dir/kubeconfig are isolated per cluster and no longer collide.
  3. kk create cluster single-cluster (default binary_dir = work_dir/kubekey) — confirm install, add node, and kk certs renew still work.

Test coverage

This is a mechanical path substitution in builtin YAML; no Go logic changed. Live-cluster verification (above) is the meaningful test and is pending.

Rollback

Plain revert of this commit restores the previous binary_dir-scoped paths. No data migration required (artifacts are regenerated on next run).

Does this PR introduce a user-facing change?

Cluster pki and kubeconfig artifacts are now stored under work_dir instead of binary_dir, allowing a single binary_dir to be shared across multiple clusters without collision.

Checklist

  • Code self-reviewed, no debug code or commented-out dead code
  • No secrets, tokens, or .env files committed
  • Commit message follows Conventional Commits
  • All commits have DCO sign-off (Signed-off-by)
  • All commits are GPG-signed (GitHub shows Verified)
  • Tests added or updated
  • Docs / CHANGELOG updated (if needed)
  • Local lint and build pass (make build)
  • Breaking changes marked above

Additional documentation, usage docs, etc.:

N/A

Notes for Reviewer

Please focus on whether any binary_dir/pki or binary_dir/kubeconfig reference was missed, and whether work_dir is guaranteed to exist/resolvable in every task context that uses these paths (it is a top-level config variable, same as binary_dir). The pki subdirectory is created implicitly by gen_cert/fetch (os.MkdirAll), so no explicit mkdir was needed.

When the same binary_dir is shared across multiple clusters (e.g. to reuse
the offline artifact package of binaries and images), the generated
{binary_dir}/pki and {binary_dir}/kubeconfig collide between clusters,
because these are cluster-specific artifacts rather than part of the
shareable offline package.

Move pki and kubeconfig to the per-cluster work_dir so that binary_dir can
stay cluster-agnostic. All affected references are local kk-host paths
(gen_cert writes, copy/fetch sources, local precheck); the certificates are
still copied to the correct remote paths (/etc/ssl/etcd/ssl,
/opt/harbor/*/ssl, /opt/docker-registry/*/ssl), so the install and
cert-renew flows are unchanged.

Signed-off-by: redscholar <blacktiledhouse@gmail.com>
@kubesphere-prow kubesphere-prow Bot added release-note kind/bug Categorizes issue or PR as related to a bug. labels Aug 19, 2026
@kubesphere-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: redscholar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubesphere-prow kubesphere-prow Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 19, 2026
@sonarqubecloud

Copy link
Copy Markdown

@redscholar redscholar added the lgtm Indicates that a PR is ready to be merged. label Aug 19, 2026
@kubesphere-prow
kubesphere-prow Bot merged commit 7dc8322 into kubesphere:main Aug 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant