Skip to content

Commit 6448517

Browse files
docs(circuit-breaker): Add comprehensive KMesh Circuit Breaker user guide
- Implement detailed documentation for Circuit Breaker feature - Include technical implementation details - Provide configuration examples and best practices - Cover troubleshooting and monitoring aspects Signed-off-by: Desh Deepak Kant <[email protected]> Closes #103
1 parent 81e68c8 commit 6448517

File tree

1 file changed

+253
-0
lines changed

1 file changed

+253
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
---
2+
draft: false
3+
linktitle: Circuit Breaker
4+
menu:
5+
docs:
6+
parent: user guide
7+
weight: 22
8+
title: Use Circuit Breaker
9+
toc: true
10+
type: docs
11+
---
12+
13+
### Preparation
14+
15+
1. Ensure KMesh is installed in your Kubernetes cluster (see [Quick Start Guide](https://kmesh.net/en/docs/setup/quickstart/))
16+
17+
2. Deploy a sample microservice application
18+
19+
3. Verify the default namespace is managed by KMesh
20+
21+
### Circuit Breaker Configuration
22+
23+
##### Deploy a Sample Application
24+
25+
Let's use a simple microservice setup to demonstrate circuit breaking:
26+
27+
```bash
28+
kubectl apply -f - <<EOF
29+
apiVersion: apps/v1
30+
kind: Deployment
31+
metadata:
32+
name: sample-service
33+
spec:
34+
replicas: 2
35+
selector:
36+
matchLabels:
37+
app: sample-service
38+
template:
39+
metadata:
40+
labels:
41+
app: sample-service
42+
spec:
43+
containers:
44+
- name: service
45+
image: your-service-image
46+
ports:
47+
- containerPort: 8080
48+
---
49+
apiVersion: v1
50+
kind: Service
51+
metadata:
52+
name: sample-service
53+
spec:
54+
selector:
55+
app: sample-service
56+
ports:
57+
- port: 80
58+
targetPort: 8080
59+
EOF
60+
```
61+
62+
##### Apply Circuit Breaker Configuration
63+
64+
Configure circuit breaker to limit connections and requests:
65+
66+
```bash
67+
kubectl apply -f - <<EOF
68+
apiVersion: kmesh.net/v1alpha1
69+
kind: CircuitBreaker
70+
metadata:
71+
name: sample-service-circuit-breaker
72+
spec:
73+
service: sample-service
74+
rules:
75+
- priority: HIGH
76+
maxConnections: 100
77+
maxPendingRequests: 50
78+
maxRequests: 200
79+
maxRetries: 3
80+
EOF
81+
```
82+
83+
### Verify Circuit Breaker Functionality
84+
85+
Use a load testing tool to simulate traffic and observe circuit breaker behavior:
86+
87+
```bash
88+
# Install hey load testing tool
89+
go install github.com/rakyll/hey@latest
90+
91+
# Generate load
92+
hey -n 1000 -c 50 http://sample-service/endpoint
93+
```
94+
95+
### Monitoring Circuit Breaker Metrics
96+
97+
Check circuit breaker metrics and logs:
98+
99+
```bash
100+
# View KMesh circuit breaker logs
101+
kubectl logs -n kmesh -l app=kmesh circuit-breaker
102+
103+
# Get circuit breaker statistics
104+
kmesh get circuit-breaker-stats sample-service
105+
```
106+
107+
### Understanding the Circuit Breaker Mechanism
108+
109+
When the circuit breaker is activated:
110+
- New connections are rejected
111+
- Existing connections are maintained
112+
- Service is protected from overload
113+
114+
### Circuit Breaker Configuration Options
115+
116+
| Parameter | Description | Default |
117+
|--------------------|--------------------------------------|------------|
118+
| priority | Request routing priority | DEFAULT |
119+
| maxConnections | Maximum concurrent connections | Unlimited |
120+
| maxPendingRequests | Maximum pending requests in queue | Unlimited |
121+
| maxRequests | Maximum concurrent requests | Unlimited |
122+
| maxRetries | Maximum retry attempts | Unlimited |
123+
124+
### Technical Implementation Details
125+
126+
##### Data Structures
127+
128+
Circuit Breaker configuration in Protocol Buffers:
129+
130+
```protobuf
131+
message CircuitBreakers {
132+
RoutingPriority priority = 1;
133+
uint32 max_connections = 2;
134+
uint32 max_pending_requests = 3;
135+
uint32 max_requests = 4;
136+
uint32 max_retries = 5;
137+
uint32 max_connection_pools = 7;
138+
}
139+
```
140+
141+
Connection Tracking Structure:
142+
143+
```c
144+
struct cluster_stats {
145+
__u32 active_connections;
146+
};
147+
148+
struct cluster_stats_key {
149+
__u64 netns_cookie;
150+
__u32 cluster_id;
151+
};
152+
```
153+
154+
##### Connection Management Workflow
155+
156+
Connection Binding Logic:
157+
158+
```c
159+
static inline int on_cluster_sock_bind(
160+
ctx_buff_t *ctx,
161+
const Cluster__Cluster *cluster
162+
) {
163+
// Check if connection limits are exceeded
164+
if (stats->active_connections >= cbs->max_connections) {
165+
// Reject connection
166+
return -1;
167+
}
168+
// Bind socket to cluster
169+
return 0;
170+
}
171+
```
172+
173+
### Advanced Configuration Example
174+
175+
```yaml
176+
apiVersion: kmesh.net/v1alpha1
177+
kind: CircuitBreaker
178+
metadata:
179+
name: complex-service-circuit-breaker
180+
spec:
181+
services:
182+
- name: service-a
183+
rules:
184+
- priority: HIGH
185+
maxConnections: 50
186+
- name: service-b
187+
rules:
188+
- priority: MEDIUM
189+
maxConnections: 100
190+
```
191+
192+
### Troubleshooting
193+
194+
##### Common Issues
195+
- Unexpected connection rejections
196+
- High error rates
197+
- Performance degradation
198+
199+
##### Debugging Steps
200+
201+
```bash
202+
# Check circuit breaker configuration
203+
kubectl describe circuitbreaker sample-service-circuit-breaker
204+
205+
# View detailed logs
206+
kubectl logs -n kmesh -l app=kmesh circuit-breaker -c circuit-breaker
207+
208+
# Check cluster status
209+
kmesh get clusters
210+
```
211+
212+
### Best Practices
213+
214+
1. Start with conservative limits
215+
2. Gradually adjust based on service performance
216+
3. Monitor circuit breaker metrics
217+
4. Use priority-based configurations
218+
219+
### Performance Considerations
220+
221+
- Kernel-native implementation
222+
- Minimal overhead
223+
- Lock-free atomic updates
224+
- eBPF map-based tracking
225+
226+
### Cleanup
227+
228+
Remove the circuit breaker and sample application:
229+
230+
```bash
231+
kubectl delete circuitbreaker sample-service-circuit-breaker
232+
kubectl delete deployment sample-service
233+
kubectl delete service sample-service
234+
```
235+
236+
### Limitations
237+
238+
- Currently focuses on TCP connections
239+
- Kernel version dependencies
240+
- Per-cluster granularity
241+
242+
### Sample Code Snippet
243+
244+
```go
245+
// Circuit Breaker Configuration Example
246+
circuitBreaker := &CircuitBreakers{
247+
Priority: RoutingPriority_HIGH,
248+
MaxConnections: 100,
249+
MaxPendingRequests: 50,
250+
MaxRequests: 200,
251+
MaxRetries: 3,
252+
}
253+
```

0 commit comments

Comments
 (0)