Skip to content

Commit 7bf045f

Browse files
shraddhabangshraddhabang
committed
[feat aga] Add documentation for AGA controller
1 parent bb18bb2 commit 7bf045f

File tree

6 files changed

+850
-0
lines changed

6 files changed

+850
-0
lines changed

‎docs/deploy/configurations.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,5 +190,6 @@ There are a set of key=value pairs that describe AWS load balancer controller fe
190190
| NLBSecurityGroup | string | true | Enable or disable all NLB security groups actions including frontend sg creation, backend sg creation, and backend sg modifications. This same behavior is able to be applied to an individual service by using the annotation `aws-load-balancer-disable-nlb-sg` |
191191
| LBCapacityReservation | string | true | Enable or disable the capacity reservation feature on ALB and NLB |
192192
| EnableTCPUDPListenerType | string | false | Enable or disable creation of TCP_UDP type listeners. This value can be overriden at the Service level by the annotation `service.beta.kubernetes.io/aws-load-balancer-enable-tcp-udp-listener` |
193+
| AGAController | string | false | Enable the Global Accelerator controller for managing AWS Global Accelerator resources through Kubernetes CRDs |
193194
| EnhancedDefaultBehavior | string | false | Enable this feature to allow the controller to remove Provisioned Capacity or mTLS settings by removing the corresponding annotation. |
194195
| EnableDefaultTagsLowPriority | string | false | If enabled, tags supplied via `--default-tags` will be overridden by tags specified in other manners, like via annotations. |

‎docs/guide/globalaccelerator/aga-controller.md‎

Lines changed: 289 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,289 @@
1+
# AWS Global Accelerator Controller
2+
3+
## Introduction
4+
5+
AWS Global Accelerator (AGA) is a networking service that improves the performance of users' traffic by up to 60% using Amazon Web Services' global network infrastructure. The AWS Load Balancer Controller (LBC) extends its capabilities with a new AGA Controller that allows users to declaratively manage AWS Global Accelerator resources using Kubernetes Custom Resource Definitions (CRDs).
6+
7+
This feature bridges a significant operational gap - traditionally, AWS Global Accelerator must be manually created and managed through the AWS console or CLI, creating operational overhead and lacking integration with Kubernetes-native workflows. The AGA Controller eliminates this gap by enabling full lifecycle management of Global Accelerator resources directly from your Kubernetes cluster.
8+
9+
## Architecture
10+
11+
The AGA Controller operates as a continuous reconciliation loop within the AWS Load Balancer Controller, watching for changes to the `GlobalAccelerator` CRD. When it detects a change, it:
12+
13+
1. Translates the desired state defined in the CRD into corresponding AWS Global Accelerator API calls
14+
2. Automatically discovers resources like Elastic Load Balancers (ELBs) referenced by Kubernetes Services, Ingresses, and Gateway APIs
15+
3. Maintains and updates the status of the AGA resources in the CRD
16+
4. Handles cleanup and resource deletion when objects are removed
17+
18+
## Key Features
19+
20+
### Monolithic CRD Design
21+
22+
The controller uses a single `GlobalAccelerator` resource to manage the entire AGA hierarchy, including:
23+
24+
- The accelerator itself
25+
- Listeners with protocol and port range configurations
26+
- Endpoint groups with region and traffic dial settings
27+
- Endpoints from different Kubernetes resource types
28+
29+
This design simplifies management by keeping all configuration in one place while still allowing granular control over each component.
30+
31+
### Full Lifecycle Management (CRUD)
32+
33+
The controller manages the complete lifecycle of AWS Global Accelerator resources:
34+
35+
- **Create**: Provisions new AWS Global Accelerator resources from CRD specifications
36+
- **Read**: Updates CRD status with current state and ARNs from AWS
37+
- **Update**: Modifies existing accelerator configurations when the CRD is updated
38+
- **Delete**: Cleans up AWS resources when the CRD is deleted
39+
40+
### Extended Endpoint Discovery
41+
42+
The AGA Controller supports automatic endpoint resolution from multiple Kubernetes resource types in the local cluster:
43+
44+
- **Service resources**: Discovers Network Load Balancers (NLBs) from Service type LoadBalancer
45+
- **Ingress resources**: Discovers Application Load Balancers (ALBs) from Ingress resources
46+
- **Gateway API resources**: Discovers ALBs and NLBs from Gateway API resources
47+
48+
### Auto-Discovery for One-Click Accelerators
49+
50+
For simple use cases, the controller offers an auto-discovery feature that automatically configures listener protocol and port ranges based on the referenced endpoints. This creates a one-click experience similar to what's available in the ELB console:
51+
52+
```yaml
53+
kind: GlobalAccelerator
54+
metadata:
55+
name: autodiscovery-accelerator
56+
namespace: test-ns
57+
spec:
58+
name: "autodiscovery-test-accelerator"
59+
listeners:
60+
- # Protocol and portRanges will be auto-discovered from the endpoint
61+
endpointGroups:
62+
- endpoints:
63+
- type: Ingress
64+
name: web-ingress
65+
weight: 200
66+
```
67+
68+
### Manual Endpoint Registration
69+
70+
For more advanced use cases, you can manually specify endpoint ARNs instead of using auto-discovery:
71+
72+
```yaml
73+
spec:
74+
listeners:
75+
- protocol: TCP
76+
portRanges:
77+
- fromPort: 443
78+
toPort: 443
79+
endpointGroups:
80+
- endpoints:
81+
- type: EndpointID
82+
endpointID: arn:aws:elasticloadbalancing:us-east-1:123456789012:loadbalancer/app/my-load-balancer/1234567890123456
83+
```
84+
85+
> **Note:** While auto-discovery currently only works within the same AWS region as the controller, you can manually specify endpoints in other regions by explicitly setting the region field and providing the endpoint ARN. For cross-region configurations, you must manually specify the protocol and port ranges as well:
86+
>
87+
> ```yaml
88+
> listeners:
89+
> - protocol: TCP # Must explicitly specify protocol
90+
> portRanges: # Must explicitly specify port ranges
91+
> - fromPort: 443
92+
> toPort: 443
93+
> endpointGroups:
94+
> - region: us-west-2 # Explicitly set a different region
95+
> endpoints:
96+
> - type: EndpointID
97+
> endpointID: arn:aws:elasticloadbalancing:us-west-2:123456789012:loadbalancer/app/my-load-balancer/1234567890123456
98+
> ```
99+
100+
### BYOIP (Bring Your Own IP) Support
101+
102+
The AWS Global Accelerator Controller supports Bring Your Own IP (BYOIP) functionality with some important constraints:
103+
104+
1. **Creation-Only Feature**: BYOIP addresses can only be specified during the initial creation of an accelerator.
105+
106+
2. **No Updates Allowed**: IP addresses cannot be updated after an accelerator has been created. If you need to change IP addresses, you must create a new accelerator.
107+
108+
3. **Handling Update Attempts**: If users attempt to modify IP addresses on an existing accelerator, the controller will log this as an invalid operation and ignore the IP address changes, preserving the original configuration.
109+
110+
For BYOIP usage, specify the addresses in the `ipAddresses` field of your GlobalAccelerator resource:
111+
112+
```yaml
113+
spec:
114+
ipAddressType: IPV4
115+
ipAddresses:
116+
- "198.51.100.10" # Your own IP from BYOIP pool
117+
```
118+
119+
### Status Reporting
120+
121+
The controller updates the CRD status with important information from the AWS Global Accelerator:
122+
123+
- The Accelerator's ARN for reference in other systems
124+
- The assigned DNS name for client access
125+
- IP address sets for both IPv4 and dual-stack configurations
126+
- The current state of the accelerator (deployed, in progress, etc.)
127+
- Conditions reflecting the health and status of the reconciliation process
128+
129+
#### Accelerator Status States
130+
131+
The `status.status` field in the GlobalAccelerator CRD reflects the current state of the accelerator in AWS. This field can have the following values:
132+
133+
- **DEPLOYED**: The accelerator is fully deployed and operational
134+
- **IN_PROGRESS**: The accelerator is being created or updated
135+
- **FAILED**: The accelerator creation or update has failed
136+
137+
When an accelerator is first created, it will typically show as IN_PROGRESS and then transition to DEPLOYED once fully provisioned. During updates, it may temporarily show as IN_PROGRESS again.
138+
139+
If the status shows FAILED, check the controller logs and the `status.conditions` field for more detailed error information.
140+
141+
## Intelligent Listener Management
142+
143+
The AGA Controller intelligently manages AWS Global Accelerator listeners to minimize service disruption during reconciliation. The controller:
144+
145+
1. Preserves existing listeners when possible to maintain stable ARNs and associated resources
146+
2. Manages conflicts carefully to ensure smooth transitions during updates
147+
3. Optimizes the operation order to minimize downtime and maintain traffic flow
148+
149+
This approach ensures reliable service while making necessary changes to your Global Accelerator configuration.
150+
151+
## Port Override Constraints and Solutions
152+
153+
AWS Global Accelerator imposes several constraints on port overrides that the controller handles automatically:
154+
155+
### Constraint 1: Listener Port Must Be Within Listener's Port Ranges
156+
157+
**Solution**: The controller validates port overrides during model building and removes invalid ones during reconciliation.
158+
159+
### Constraint 2: Endpoint Port Can't Overlap With ANY Listener Port Ranges
160+
161+
**Solution**: Cross-validation checks endpoint ports against all listener port ranges and removes overlapping configurations.
162+
163+
### Constraint 3: No Duplicate Endpoint Ports Across Listeners
164+
165+
**Solution**: The controller tracks all endpoint ports across the accelerator to ensure uniqueness.
166+
167+
## Sample CRDs
168+
169+
### Basic Global Accelerator with Single TCP Port
170+
171+
```yaml
172+
apiVersion: aga.k8s.aws/v1beta1
173+
kind: GlobalAccelerator
174+
metadata:
175+
name: simple-accelerator
176+
namespace: default
177+
spec:
178+
name: "simple-accelerator"
179+
ipAddressType: IPV4
180+
tags:
181+
Environment: "production"
182+
listeners:
183+
- protocol: TCP
184+
portRanges:
185+
- fromPort: 80
186+
toPort: 80
187+
clientAffinity: NONE
188+
```
189+
190+
### Multiple Port Ranges with Source IP Affinity
191+
192+
```yaml
193+
apiVersion: aga.k8s.aws/v1beta1
194+
kind: GlobalAccelerator
195+
metadata:
196+
name: multi-port-accelerator
197+
namespace: default
198+
spec:
199+
ipAddressType: IPV4
200+
listeners:
201+
- protocol: TCP
202+
portRanges:
203+
- fromPort: 80
204+
toPort: 80
205+
- fromPort: 443
206+
toPort: 443
207+
clientAffinity: SOURCE_IP
208+
```
209+
210+
### Complex Configuration with Multiple Listeners and Endpoints
211+
212+
```yaml
213+
apiVersion: aga.k8s.aws/v1beta1
214+
kind: GlobalAccelerator
215+
metadata:
216+
name: complex-accelerator
217+
namespace: default
218+
spec:
219+
ipAddressType: IPV4
220+
tags:
221+
Environment: "test"
222+
Team: "platform"
223+
listeners:
224+
- protocol: TCP
225+
portRanges:
226+
- fromPort: 80
227+
toPort: 80
228+
- fromPort: 443
229+
toPort: 443
230+
clientAffinity: SOURCE_IP
231+
endpointGroups:
232+
- trafficDialPercentage: 100
233+
portOverrides:
234+
- endpointPort: 8081
235+
listenerPort: 80
236+
endpoints:
237+
- type: Service
238+
name: service-nlb-1
239+
weight: 128
240+
clientIPPreservationEnabled: true
241+
- type: Ingress
242+
name: test-ingress
243+
weight: 240
244+
- protocol: UDP
245+
portRanges:
246+
- fromPort: 53
247+
toPort: 53
248+
clientAffinity: NONE
249+
```
250+
251+
## Troubleshooting
252+
253+
Common issues and their solutions:
254+
255+
### Status Shows Failure
256+
257+
Check the controller logs for detailed error messages:
258+
259+
```bash
260+
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
261+
```
262+
263+
### Endpoint Discovery Not Working
264+
265+
Ensure your Service/Ingress/Gateway resources:
266+
267+
1. Are properly configured with correct annotations
268+
2. Have been successfully provisioned with actual AWS load balancers
269+
3. Are in the same namespace as specified in the endpoint
270+
271+
### Port Override Conflicts
272+
273+
If port overrides are being automatically removed, check for these common issues:
274+
275+
1. Listener port not within listener's port ranges
276+
2. Endpoint port overlapping with a listener port range
277+
3. Duplicate endpoint ports across different listeners
278+
279+
## Current Limitations and Future Enhancements
280+
281+
### Cross-Namespace Reference Limitations
282+
283+
The initial release of the AWS Global Accelerator Controller does not support cross-namespace endpoint references. This means that all endpoint resources (Services, Ingresses, Gateways) must be in the same namespace as the GlobalAccelerator resource that references them.
284+
285+
## References
286+
287+
- [AWS Global Accelerator Documentation](https://docs.aws.amazon.com/global-accelerator/latest/dg/what-is-global-accelerator.html)
288+
- [GlobalAccelerator CRD Reference](spec.md)
289+
- [AWS Load Balancer Controller Installation](../../deploy/installation.md)

0 commit comments

Comments
 (0)