TraceMyPods is a Kubernetes-native DevOps Focused (chat WITH OllamaAI) token-gated service platform that leverages a multiple microservices architecture with secure Istio service mesh, GPU acceleration, and a robust monitoring stack with full Terraform and Helm packaged to deploy on AWS EKS. It provides a seamless user experience for AI interactions, token management, and advanced analytics.
Application Workflow (dev) : Click_To_View
Istio Service Mesh | Jaeger | AWS EKS | GPU Nodes | Grafana | Loki | Prometheus
HELM | Terraform | Trevy | Github Actions | Falco (Security) | Kyverno (PSPolicy) | Kube-Hunter (CIS Benchmark) |
IstioGateway/VirtualService | Ingress | HPA | VPA | Pod Disruption Budget | Network Policy | Resource Quotas | SecurityContext | PV, Storage Class, PVC, EBS | PodSecurityPolicies | RBAC |
ConfigMaps / Secrets | Taints & Tolerations | Readiness & Liveness Probes
Deployments / CronJobs | Affinity / Anti-Affinity | Kiali | Kube-Hunter | Minikube (local)
Kafka | Redis | MongoDB | Ollama (AI) | Node.js | Python | GO |VectorDB (qdrant) | Microservices | Payments (razorpay) | Invoice/Reports | EMAIL | PostmanCollection | LoadTesting | OpenTelemetry | Gemini_AI | S3 (File Browser) | Image Generation API
- Landing page where users can explore the platform features, chat with AI models, and access various services.
- Chat interface for users to interact with premium AI models, powered by Ollama.
- Chat interface for users to interact with free AI models, providing a seamless user experience.
- Image generation API that allows users to create images based on text prompts, integrated with Cloudflare AI Workers.
- Features overview showcasing the capabilities of the TraceMyPods platform, including AI interactions, token management, and advanced analytics.
- Model purchase interface where users can select models and proceed with payment.
- Payment success page confirming the successful transaction and model activation.
- Card details page for entering payment information securely.
- Payment confirmation page for reviewing and confirming payment details.
- Payment successful page confirming the completion of the transaction.
- Preview of the invoice generated after a successful purchase. (sent to user email and stored in S3 bucket)
- Preview of the email sent to users upon successful purchase, containing invoice/token details and confirmation.
Admin dashboard to view purchase history, manage free and paid tokens and earnings and export business purchase data.
- Here we can view and search invoices stored in S3 Bucket
- Here we can view order history, earnings and export business purchase data including the active tokens and free tokens
- OtelAPI for code-level tracing and AI integration debugging, providing insights into application performance.
- To view the service mesh topology, traffic flow, and health status of microservices.
- To visualize metrics collected from the Kubernetes cluster and applications.
- Viewing and searching logs from various microservices using Loki.
- Distributed tracing for microservices using Jaeger.
- Scanning container images and Kubernetes clusters for vulnerabilities using Trivy.
- Real-time monitoring and detection of security threats in running containers using Falco.
- Active reconnaissance tool for Kubernetes clusters to identify potential security issues.
- Enforcing security policies for Kubernetes pods using Kyverno.
- GitOps to deploy tracemypods and kafka in Kubernetes.
- Monitoring GPU utilization and performance metrics using CloudWatch.
- Secrets management and data protection for sensitive information.
This architecture diagram illustrates the deployment of the TraceMyPods application on AWS EKS, showcasing the integration of Istio service mesh, GPU nodes, and various microservices communication and workflows. It highlights the use of Istio for secure service communication, Jaeger for distributed tracing, and the overall infrastructure setup including monitoring and security components.
- EKS Cluster: The Kubernetes cluster where the TraceMyPods application is deployed. This cluster is configured with GPU nodes for AI workloads.
- Istio Gateway: Manages ingress traffic, routing to different APIs and load balancing. IstioGateway.yaml
- Istio Service Mesh: Enables secure, observable communication between microservices, with features like mutual TLS, traffic management, and telemetry. Istio
- Kiali: Visualizes and manages the service mesh.
- Jaeger: Jaeger with Otelapi used to display distributed traces for microservices, helping to identify performance bottlenecks and trace requests across services with
AIintegration. Jaeger - Kubecost: Integrates for real-time cost monitoring and optimization. (inprogress)
- ALB Ingress Controller: Handles external traffic routing via AWS Application Load Balancer. Ingress.tf
- HPA & VPA: Horizontal and Vertical Pod Autoscalers for dynamic scaling based on resource usage.
- PodDisruptionBudget: Maintains application availability during node updates or disruptions.
- Resource Quotas: Enforces resource limits per namespace.
- SecurityContext: Applies security settings at the pod and container level.
- ConfigMaps & Secrets: Manages configuration and sensitive data.
- Taints & Tolerations: Controls pod scheduling on specific nodes.
- Readiness & Liveness Probes: Ensures pod health and availability.
- NetworkPolicy: Restricts and controls pod-to-pod communication.
- Affinity & Anti-Affinity: Optimizes pod placement for reliability and performance.
- Persistent Storage: Uses
StorageClassto dynamically provision EBS volumes for PVCs. - IRSA: Implements IAM Roles for Service Accounts for secure AWS service access (e.g., S3).
- GPU Nodes: Supports AI workloads with NVIDIA T4 GPUs (
g4dn.xlarge).tolerations: - key: "gpu" operator: "Equal" value: "true" effect: "NoSchedule"
This architecture diagram illustrates the AWS infrastructure setup for the TraceMyPods application, including EKS, VPC with Public and Private Subnets, NAT Gateway, S3, SES, Nodes and other AWS services. It highlights the use of Terraform for Infrastructure as Code (IAC) to provision and manage resources.
IAC Stack | TF
Terraform to provision AWS resources for the TraceMyPods application, including EKS, VPC, IAM roles, S3 buckets, and more. This support reusable modules to provision infrastructure components.
Terraform modules include:
- EKS Cluster with GPU nodes
- VPC with Public and Private Subnets
- IAM roles for EKS and IRSA
- S3 bucket for file storage
- NAT Gateway for internet access in private subnets
- Cloudflare for DNS and CDN
- Security Groups for network access control
- ALB Ingress Controller for traffic routing
- Global Accelerator for improved application availability and performance
- Helm Installation of (Istio, Kiali, Prometheus, Loki, Grafana, falco, Kafka)
This Stack includes Prometheus for metrics collection, Loki for log aggregation, and Grafana for visualization. It provides comprehensive monitoring and alerting capabilities for the TraceMyPods application. Monitoring
This Stack includes security measures such as Kube-Hunter for vulnerability scanning, Falco for runtime security monitoring, Kyverno for PodSecurityPolicy enforcement and Trivy for container image vulnerability scanning and Code Scanning. It ensures the TraceMyPods application is secure and compliant with best practices.
Tools used:
- Kube-Hunter for vulnerability scanning | docs
- Falco for runtime security monitoring | docs
- Kyverno for PodSecurityPolicy enforcement | docs
- GitHub Actions for CI to automate Trivy for container image vulnerability scanning and code scanning | docs | CI | Code Scanning
- Kube-Bench for CIS Benchmark compliance | docs
- Vault and Kubernetes Secrets for sensitive data management | docs
- Network Policies for pod communication control | NP
This Stack includes ArgoCD for GitOps, Helm for package management, and Terraform for Infrastructure as Code (IAC). It supports canary and blue-green deployments and provides a robust deployment strategy for the TraceMyPods application.
Tools used:
- ArgoCD for GitOps | docs
- Helm for package management | HELM
- IAC with Terraform | TF
- Canary, Rolling, and Blue-Green deployments in YAML files
- PriorityClass for pod priority scheduling | PriorityClass
This Stack includes various tools and technologies used in the TraceMyPods application, such as Istio for service mesh, Jaeger for distributed tracing, Prometheus for metrics collection, Loki for log aggregation, and Grafana for visualization. It provides a comprehensive DevOps stack for the TraceMyPods application.
- Service mesh for secure, observable microservices | docs
- Istio Gateway for traffic routing | IstioConfig.yaml
- Kiali for service mesh observability
- Distributed tracing for microservices | Jaeger.yaml
- Otelapi for code level tracing and AI integration debugging| otelapi | oteldashboard
- Cluster Security scanning for vulnerabilities | text
- Runtime security monitoring | text
- Container image vulnerability scanning. text
- Cluster security checks text
- Github Actions for CI/CD for trivy code scanning ( text )
- CI pipeline to build docker image (x86 and ARM) and push to hub.docker.com | CI
- Kubernetes package management for deployments | HelM
- Postman collection for API testing and interaction with the TraceMyPods application. LoadTest/postman_collection.js
- Load testing the application using Grafana K6 to ensure performance and scalability. LoadTest/k6_API-test.js
- KubeCost for cost monitoring
- Spot Intance for cost optimization
- AWS API Gateway for API management
- EKS RBAC for fine-grained access control
- Litmus chaos engineering
- EFK stack (Elasticsearch, Fluentbit, Kibana) or AWS OpenSearch (Remove loki logs and use EFK stack)
| Topic | Resource |
|---|---|
| CKA | Kubernetes Official Docs, Killer.sh CKA |
| CKAD | KodeKloud CKAD, Katacoda Scenarios |
| CKS | Practical CKS Guide, Sysdig Threat Hunting |
| Istio | Istio.io Docs |
| Monitoring | Awesome Prometheus Alerts |
| GitOps | ArgoCD Docs, Weave GitOps |
| EKS Best Practices | AWS EKS Workshop |
Ahmad Raza
Sr. DevOps Engineer | Cloud Infra Specialist
🔗 ahmadraza.in
🔗 linkedin.com/in/ahmad-raza-devops
For more, visit ahmadraza.in
Detailed commands, manifests, and guides are available on my blog.