Proposal: Support Custom CA Certificate for Karmada Instance in Operator

docs/proposals/karmada-operator/custom_ca_cert/README.md

@@ -0,0 +1,123 @@
+---
+title: Support Custom CA Certificate for Karmada Control Plane
+authors:
+- "@jabellard"
+reviewers:
+- "@RainbowMango"
+approvers:
+- "@RainbowMango"
+
+creation-date: 2024-11-07
+
+---
+
+# Support Custom CA Certificate for Karmada Control Plane
+
+## Summary
+
+This proposal aims to extend the Karmada operator with support for custom CA certificates in managed Karmada control planes. By allowing users to provide their own CA certificate,
+this feature enables multi-data center redundancy for critical Karmada deployments and aligns with organizational policies around disaster recovery (DR). This capability
+is especially important for cases where a managed Karmada control plane is stretched across multiple management clusters that span multiple data centers and shares an underlying etcd instance, requiring the control plane instances to have a unified CA.
+
+## Motivation
+
+In high-availability scenarios where Karmada control planes power mission-critical workloads, ensuring that each managed control plane adheres to strict organizational policies around availability
+and disaster recovery is essential. Deploying control planes across multiple management clusters that span multiple data centers provides redundancy and resilience in the event of a data center outage. 
+However, when multiple control plane instances must access a shared etcd instance and present a unified API endpoint, using a common CA certificate across instances is necessary as the same CA should be used to verify client certificates. 
+By enabling users to specify a custom CA certificate, this feature ensures that control planes spanning data centers function as a cohesive unit while meeting security and availability standards.
+
+### Architecture Overview
+![architecture-overview](./architecture-overview.png)
+
+### Goals
+
+- Allow users to specify a custom CA certificate which is used to sign the Karmada API Server certificate and for verifying client certificates.
+- Ensure that control plane instances deployed across multiple management clusters that span multiple data centers use the same CA, enabling secure and seamless cross-data center communication.
+- Enable operators to align Karmada control plane PKI with organizational policies for high availability and security.
+
+### Non-Goals
+
+- Change the default behavior of the Karmada operator when no custom CA is provided.
+- Alter the process of how control planes are connected to etcd.
+- Address custom CA requirements for other concerns such as PKI for an in-cluster ETCD instance or front proxy communications.
+
+## Proposal
+
+The proposal introduces a new optional field, `CertificateConfig`, in the `KarmadaSpec`, where users can specify a custom CA certificate for the Karmada control plane. 
+This configuration will allow users to choose between `Auto` and `Custom` modes:
+
+1. *Auto Mode*: The operator will automatically generate a CA certificate for the control plane. This mode is the default.
+2. *Custom Mode*: The user can specify a Kubernetes secret containing their CA certificate and key to be used for the control plane.
+
+### API Changes
+
+```go
+// CertificateMode defines the mode for managing certificates for the Karmada control plane.
+type CertificateMode string
+
+const (
+   // Auto mode means the certificate will be automatically generated by the operator.
+   Auto CertificateMode = "Auto"
+
+   // Custom mode means the user provides their own CA certificate.
+   Custom CertificateMode = "Custom"
+)
+
+// CustomCertConfig defines the structure for user-provided CA certificates in Custom mode.
+type CustomCertConfig struct {
+   // KarmadaCACert references a Kubernetes secret containing the CA certificate for the Karmada instance.
+   // The secret must contain the following data keys:
+   // tls.crt: The TLS certificate.
+   // tls.key: The TLS private key.
+   // +optional
+   KarmadaCACert *LocalSecretReference `json:"karmadaCaCert,omitempty"`
+}
+
+// CertificateConfig is the configuration for managing certificates for the Karmada control plane.
+type CertificateConfig struct {
+   // Mode defines how certificates for the Karmada control plane should be managed.
+   // Valid values are "Auto" and "Custom". Defaults to "Auto".
+   // +kubebuilder:validation:Enum=Auto;Custom
+   // +kubebuilder:default=Auto
+   Mode CertificateMode `json:"mode,omitempty"`
+
+   // Custom contains the configuration for user-provided certificates when in Custom mode.
+   // +optional
+   Custom *CustomCertConfig `json:"custom,omitempty"`
+}
+
+// KarmadaSpec is the specification of the desired behavior of the Karmada.
+type KarmadaSpec struct {
+   // CertificateConfig manages the certificate settings for the Karmada control plane.
+   CertificateConfig *CertificateConfig `json:"certificateConfig,omitempty"`
+
+   // (other fields omitted for brevity)
+}
+```
+### User Stories
+
+#### Story 1
+As a cloud infrastructure architect, I want to deploy a highly available managed Karmada control plane across multiple management clusters that span multiple data centers to meet disaster recovery requirements. 
+By deploying the managed control plane instances using this architecture and configuring them to use the same CA certificate and underlying etcd instance, I can ensure the stretched instance can be securely accessed via a unified, 
+load-balanced API endpoint. This setup provides resilience in case of data center outages, adhering to DR requirements and minimizing the risk of service disruption.
+
+
+### Risks and Mitigations
+
+1. *Incorrect Secret Format*: If the provided CA certificate secret does not follow the required format, the control plane setup may fail.
+
+    - *Mitigation*: The operator will validate the format of the provided secret, ensuring it adheres to the expected format, and return detailed error messages if the configuration is incorrect.
+
+2. *Backward Compatibility*: Introducing a custom CA feature might impact users who do not need or configure this option.
+
+    - *Mitigation*: This feature is fully optional; if no CA is provided, the operator will default to generating one as it currently does, maintaining backward compatibility.
+
+## Design Details
+
+The `CertificateConfig` in `KarmadaSpec` will allow users to specify the reference to a Kubernetes secret that contains a custom CA certificate. During the reconciliation process, the Karmada operator will:
+
+Check if `CertificateConfig.Mode` is set to `Custom`.
+If `Custom`, retrieve the certificate from the secret specified in `CertificateConfig.Custom.CACert` and use it as the control plane’s CA certificate.
+If `Auto`, generate a CA certificate as per the operator's default behavior.
+
+This feature requires minimal changes to the reconciliation process and does not impact existing installations that do not specify a custom CA.