Skip to content

Conversation

@ramessesii2
Copy link

@ramessesii2 ramessesii2 commented Oct 10, 2025

Implements a new clusterctl migrate command to convert core CAPI resources i.e., cluster.x-k8s.io resources between API versions, currently supporting v1beta1 → v1beta2 migration.

✅ Tests
✅ Docs

Implements #12716

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 10, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign justinsb for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the do-not-merge/needs-area PR is missing an area label label Oct 10, 2025
@k8s-ci-robot
Copy link
Contributor

Welcome @ramessesii2!

It looks like this is your first PR to kubernetes-sigs/cluster-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 10, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @ramessesii2. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 15, 2025
@ramessesii2 ramessesii2 changed the title feat: add clusterctl alpha migrate command feat: add experimental clusterctl migrate command Oct 15, 2025
@ramessesii2 ramessesii2 force-pushed the RAMESSES/migrate-cmd branch 3 times, most recently from 6f90e35 to c2050f4 Compare October 15, 2025 08:51
@ramessesii2 ramessesii2 marked this pull request as ready for review October 15, 2025 08:52
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 15, 2025
@ramessesii2 ramessesii2 changed the title feat: add experimental clusterctl migrate command ✨ feat: add experimental clusterctl migrate command Oct 15, 2025
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did a quick first pass on migrate.go

please keep the commits squashed to 1 on amends.

"os"

"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/runtime/schema"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add new line between these two.

k8s vs non-k8s sources.

Copy link
Author

@ramessesii2 ramessesii2 Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently, linter is not happy with the suggested change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently, linter is not happy with the suggested change.

that's odd. linters normally allow you to define as many 'groups' of imports as you like.

@sbueringer
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 16, 2025
    - Experimental migration support focuses on v1beta1 to v1beta2
    conversions for core Cluster API resources

Signed-off-by: Satyam Bhardwaj <[email protected]>
@fabriziopandini
Copy link
Member

/area clusterctl

@k8s-ci-robot k8s-ci-robot added area/clusterctl Issues or PRs related to clusterctl and removed do-not-merge/needs-area PR is missing an area label labels Oct 16, 2025
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the updates. i did a bigger pass on the code today.
this is in relatively good shape, but i made some suggestions to re-organize it a bit and make api types opaque to the migrate package.

var migrateOpts = &migrateOptions{}

var supportedTargetVersions = []string{
clusterv1.GroupVersion.Version,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't there a programmatic way to enumerate these instead of requiring the OWNERS to manually add a new version here?

e.g. core k8s types manage that in the internal types for a given group, but in CAPI the layout is different.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I don't think it is necessary to automate managing this list, I would prefer to avoid unnecessary complexity

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list should be defined only once, and it should be defined into the library, not at the cmd level


func runMigrate(args []string) error {
if !isSupportedTargetVersion(migrateOpts.toVersion) {
return errors.Errorf("invalid --to-version value %q: supported versions are %s", migrateOpts.toVersion, strings.Join(supportedTargetVersions, ", "))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return errors.Errorf("invalid --to-version value %q: supported versions are %s", migrateOpts.toVersion, strings.Join(supportedTargetVersions, ", "))
return errors.Errorf("invalid --to-version value %q. Supported versions are: %s", migrateOpts.toVersion, strings.Join(supportedTargetVersions, ", "))


func init() {
migrateCmd.Flags().StringVarP(&migrateOpts.output, "output", "o", "", "Output file path (default: stdout)")
migrateCmd.Flags().StringVar(&migrateOpts.toVersion, "to-version", clusterv1.GroupVersion.Version, fmt.Sprintf("Target API version for migration (supported: %s)", strings.Join(supportedTargetVersions, ", ")))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
migrateCmd.Flags().StringVar(&migrateOpts.toVersion, "to-version", clusterv1.GroupVersion.Version, fmt.Sprintf("Target API version for migration (supported: %s)", strings.Join(supportedTargetVersions, ", ")))
migrateCmd.Flags().StringVar(&migrateOpts.toVersion, "to-version", clusterv1.GroupVersion.Version, fmt.Sprintf("Target API version for migration. Supported versions are: %s)", strings.Join(supportedTargetVersions, ", ")))

parser := migrate.NewYAMLParser(scheme.Scheme)

targetGV := schema.GroupVersion{
Group: clusterv1.GroupVersion.Group,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is ok, but it hardcodes the group to the group of an imported version package.
i think a local constant that copies the group string might be better.
...or a proper mapping that takes the user version input and determines the group from it.

"os"

"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/runtime/schema"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently, linter is not happy with the suggested change.

that's odd. linters normally allow you to define as many 'groups' of imports as you like.

yamlserializer "k8s.io/apimachinery/pkg/runtime/serializer/yaml"
yamlutil "k8s.io/apimachinery/pkg/util/yaml"

clusterv1 "sigs.k8s.io/cluster-api/api/core/v1beta2"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto. the GVKs must be opaque to the internal/migrate package i would say. it can accept a list of known source and target GVKs.


const (
// ResourceTypeCoreV1Beta1 identifies v1beta1 core ClusterAPI resources.
ResourceTypeCoreV1Beta1 ResourceType = iota
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the classification of input GVKs can be done based on provided GVK inputs as params.
the first const in the enum here could be a generic ResourceTypeSupported (or similar for a type that can be parsed on migrated).


func (p *YAMLParser) classifyResource(gvk schema.GroupVersionKind) ResourceType {
if p.isCoreCAPIGroup(gvk.Group) {
if gvk.Version == "v1beta1" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best to check from the input GVK i mention above instead of hardcoding known versions in string literals.

return ok
}

var coreCapiGroups = map[string]struct{}{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put vars on top of the doc.


Examples:
# Migrate from file to stdout
clusterctl migrate cluster.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would migrate from new to old be supported?
e.g. v1beta2 -> v1beta1

Copy link
Member

@fabriziopandini fabriziopandini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ramessesii2 thanks for this PR and sorry for the delay in the review.

Two main comments from my side

  • The code should be structured to support usage of clusterctl as a library
  • I would suggest to simplify the code as much as possible, the code target a specific use case and we do not expect to change this code frequently (let's prefer simplicity over extensibility)

See comments for more details; also feel free to pin me in the CAPI channel if you need more clarifications

Comment on lines +45 to +48
var migrateCmd = &cobra.Command{
Use: "migrate [SOURCE]",
Short: "EXPERIMENTAL: Migrate cluster.x-k8s.io resources between API versions",
Long: `EXPERIMENTAL: Migrate cluster.x-k8s.io resources between API versions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we should position this command as "clusterctl config migrate" instead of "clusterctl migrate", for a ca reasons:

  • as of today the "config" subcommand in clusterctl supervise to repositories, and repositories is where clusterctl expect the yaml templates to be.
  • it will allow to position this command in a way that makes more sense to users, e.g. description could be "Migrate cluster and cluster class templates between API versions" (which is less opaque than Migrate cluster.x-k8s.io resources")
  • it will keep the top level list of commands in clusterctl nice and clean (and we will not add an experimental command to it)
  • it will be nicely consistent with kubeadm config migrate

Copy link
Member

@sbueringer sbueringer Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wdyt is this enough to make clear that we are just converting YAML files?

I saw so much confusion already about how apiVersions work that I wonder if that command will make some folks think they have to run the migration when bumping to v1beta2.

Especially as we have components like CRD migrator in CAPI

Should we rather use conversion instead of migration? (It's using our conversion and not our migration code, also apiserver calls this conversion)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clusterctl convert yaml also works for me (e.g we already have clusterctl generate yaml)

Copy link
Member

@sbueringer sbueringer Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if true, but according to Gemini convert would probably be a good fit

image

var migrateOpts = &migrateOptions{}

var supportedTargetVersions = []string{
clusterv1.GroupVersion.Version,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list should be defined only once, and it should be defined into the library, not at the cmd level

"k8s.io/apimachinery/pkg/runtime/schema"

clusterv1 "sigs.k8s.io/cluster-api/api/core/v1beta2"
"sigs.k8s.io/cluster-api/cmd/clusterctl/internal/migrate"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The migrate command should not be implemented in an internal packages.

All clusterctl commands should be implemented as a single method in the clusterctl Client interface so they can be used both from the cmd as well as from users leveraging to clusterctl as a library

This also implies that most of the content of runMigrate must be moved into the method above.

The single method in the clustectl library can then use subcomponents from client subpacages.
For this command I will be ok to have the content of internal/migrate moved to client/migrate

return false
}

func runMigrate(args []string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the code in this func must be moved to the client interface method, all except:

  • flag validation
  • input handling (reading from a file or stdin)
  • output handling (writing to file, stdout, stderr)

}

func (p *YAMLParser) classifyResource(gvk schema.GroupVersionKind) ResourceType {
if p.isCoreCAPIGroup(gvk.Group) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's simplify this, there is only one CAPI group

Suggested change
if p.isCoreCAPIGroup(gvk.Group) {
if gvk.Group == clusterv1.GroupVersion.Group {

func (c *Converter) ConvertResource(info ResourceInfo, obj runtime.Object) ConversionResult {
gvk := info.GroupVersionKind

if gvk.Group == clusterv1.GroupVersion.Group && gvk.Version == c.targetGV.Version {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I wrong or this should never happen considering we are calling func this only when dealing with ResourceTypeCoreV1Beta1 objects?

If this is the case, either drop or return an error in this case
(the warning is not actionable for users, because it is surfacing an error in the code)

}
}

if gvk.Group != clusterv1.GroupVersion.Group {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I wrong or this should never happen considering we are calling func this only when dealing with ResourceTypeCoreV1Beta1 objects?

}

targetGVK, err := c.getTargetGVK(gvk)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I wrong or this should never happen considering we are calling func this only when dealing with ResourceTypeCoreV1Beta1 objects?

}

// Use scheme-based conversion for all remaining cases
convertedObj, err := c.scheme.ConvertToVersion(obj, targetGVK.GroupVersion())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say we should fail hard if we are dealing with a core object that does not support convertible

type gvkConversionMap map[schema.GroupVersionKind]schema.GroupVersionKind

// ConversionResult represents the outcome of converting a single resource.
type ConversionResult struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also in this case, either we convert or we fail hard.
(we can drop Converted, Error, Warnings, and most probably we can use the Object directly as a input to the convert func)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/clusterctl Issues or PRs related to clusterctl cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants