Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structured AuthorizationConfiguration #11852

Conversation

chadswen
Copy link
Member

@chadswen chadswen commented Jan 4, 2025

What type of PR is this?
/kind feature

What this PR does / why we need it:
Adds the ability to configure the Kubernetes API server with a structured authorization configuration file.

Structured AuthorizationConfiguration is a new feature in Kubernetes v1.29+ (GA in v1.32) that configures the API server's authorization modes with a structured configuration file. AuthorizationConfiguration files offer features not available with the --authorization-mode flag, although Kubespray supports both methods and authorization-mode remains the default for now.

Because the --authorization-config and --authorization-mode flags are mutually exclusive, the authorization_modes ansible variable is ignored when kube_apiserver_use_authorization_config_file is set to true. The two features cannot be used at the same time.

Docs: https://kubernetes.io/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file
Blog + Examples: https://kubernetes.io/blog/2024/04/26/multi-webhook-and-modular-authorization-made-much-easier/
KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3221-structured-authorization-configuration

Not to be confused with Structured Authentication, which is tracked in #11834. This is an authz feature, the other issue is authn.

Special notes for your reviewer:
I tested this all the way back to k8s v1.29 when AuthorizationConfiguration was first introduced as an alpha feature, although v1.29 required some additional workarounds with kubeadm_patches, which I included in example comments.

I also included some example comments with CEL expressions that allowed me to configure webhook authorizers without hitting kubeadm 1.29+ issues that block cluster creation and upgrades such as this one: kubernetes/cloud-provider-openstack#2575 . My workaround configures the webhook to ignore requests from kubeadm and system components, which prevents fatal errors from webhooks that are not available yet, and should be authorized by Node or RBAC anyway.

Does this PR introduce a user-facing change?:

[kubernetes/control-plane] Added support for [structured AuthorizationConfiguration files](https://kubernetes.io/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file).

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 4, 2025
@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 4, 2025
@chadswen chadswen force-pushed the structured-authorization-config-file branch from 8876c39 to 12e0f3a Compare January 4, 2025 00:38
@chadswen
Copy link
Member Author

chadswen commented Jan 4, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jan 4, 2025
@chadswen
Copy link
Member Author

chadswen commented Jan 4, 2025

/retest
/retest-failed

@chadswen chadswen changed the title Structured AuthorizationConfig Structured AuthorizationConfiguration Jan 4, 2025
Adds the ability to configure the Kubernetes API server with a structured authorization configuration file.

Structured AuthorizationConfiguration is a new feature in Kubernetes v1.29+ (GA in v1.32) that configures the API server's authorization modes with a structured configuration file.
AuthorizationConfiguration files offer features not available with the `--authorization-mode` flag, although Kubespray supports both methods and authorization-mode remains the default for now.

Note: Because the `--authorization-config` and `--authorization-mode` flags are mutually exclusive, the `authorization_modes` ansible variable is ignored when `kube_apiserver_use_authorization_config_file` is set to true. The two features cannot be used at the same time.

Docs: https://kubernetes.io/docs/reference/access-authn-authz/authorization/#configuring-the-api-server-using-an-authorization-config-file
Blog + Examples: https://kubernetes.io/blog/2024/04/26/multi-webhook-and-modular-authorization-made-much-easier/
KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3221-structured-authorization-configuration

I tested this all the way back to k8s v1.29 when AuthorizationConfiguration was first introduced as an alpha feature, although v1.29 required some additional workarounds with `kubeadm_patches`, which I included in example comments.

I also included some example comments with CEL expressions that allowed me to configure webhook authorizers without hitting kubeadm 1.29+ issues that block cluster creation and upgrades such as this one: kubernetes/cloud-provider-openstack#2575.
My workaround configures the webhook to ignore requests from kubeadm and system components, which prevents fatal errors from webhooks that are not available yet, and should be authorized by Node or RBAC anyway.
@chadswen chadswen force-pushed the structured-authorization-config-file branch from 12e0f3a to c0fb5b1 Compare January 6, 2025 18:46
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 6, 2025
@chadswen chadswen requested a review from VannTen January 6, 2025 20:09
@VannTen
Copy link
Contributor

VannTen commented Jan 7, 2025

Thanks, looks great !
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 7, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chadswen, VannTen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 7, 2025
@VannTen
Copy link
Contributor

VannTen commented Jan 7, 2025

Possible follow-up : validation of the authz_config with ansible.utils.validate.
This would need to extract the schema from the go structure in k/k, as there is now openapi/jsonschema produced by upstream for this.
See the slack thread (about authn, but would apply here as well I think) : https://kubernetes.slack.com/archives/C0EN96KUY/p1736158568571259

@k8s-ci-robot k8s-ci-robot merged commit 8443f37 into kubernetes-sigs:master Jan 7, 2025
42 checks passed
@chadswen chadswen deleted the structured-authorization-config-file branch January 7, 2025 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants