Target

• Build two Docker images (node + orders) inside Minikube’s Docker.

• Install ingress-nginx via Helm with WAF enabled.

• Generate certs: a CA, a server cert for test.com, and a client cert.

• Create k8s secrets for TLS + client CA.

• install the app chart.

• Map test.com → Minikube IP in /etc/hosts (this simulates owning the domain).

• Test with/without the client cert to see mTLS enforcement, rate limits, and WAF in action.

Prerequisite: install docker

brew install minikube

brew install helm

Provide at least 8GB to docker in docker desktop settings

minikube start --cpus=4 --memory=8g

Helm + K8s POC: Ingress + mTLS + WAF (Minikube)

This POC has two tiny APIs, a full Helm chart to deploy them with 2 replicas each, and an NGINX Ingress with TLS termination, mutual TLS (mTLS) client auth, rate limiting, and WAF (ModSecurity + OWASP CRS). It is designed for Minikube.

Stack: Node.js (Express) apps → ClusterIP Services → NGINX Ingress Controller (Helm) → Ingress (TLS + mTLS + WAF + rate limit)

---

0) Directory Layout

helm-k8s-poc/
├── apps/
│   ├── node/
│   │   ├── package.json
│   │   ├── server.js
│   │   └── Dockerfile
│   └── orders/
│       ├── package.json
│       ├── server.js
│       └── Dockerfile
├── helm/
│   ├── poc-apps/
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   │   └── templates/
│   │       ├── deployment-node.yaml
│   │       ├── service-node.yaml
│   │       ├── deployment-orders.yaml
│   │       ├── service-orders.yaml
│   │       └── ingress.yaml
│   └── ingress-nginx/
│       └── ingress-nginx-waf.yaml
└── scripts/
    └── gen-certs.sh

---

make directory structure

mkdir -p helm-k8s-poc/{apps/{node,orders},helm/{poc-apps/templates,ingress-nginx},scripts}

# then
cd helm-k8s-poc

Create the following files , or just clone my project

1) Applications (Node.js)

apps/node/package.json

{
  "name": "poc-node",
  "version": "0.1.0",
  "main": "server.js",
  "license": "MIT",
  "dependencies": {
    "express": "^4.19.2"
  }
}

apps/node/server.js

const express = require('express');
const app = express();
const PORT = process.env.PORT || 3000;

app.get('/healthz', (req, res) => res.send('ok'));
app.get('*', (req, res) => {
  res.json({
    service: 'node',
    path: req.path,
    headers: req.headers
  });
});

app.listen(PORT, () => console.log(`node service on :${PORT}`));

apps/node/Dockerfile

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm install --production || npm install
COPY server.js ./
EXPOSE 3000
CMD ["node","server.js"]

---

apps/orders/package.json

{
  "name": "poc-orders",
  "version": "0.1.0",
  "main": "server.js",
  "license": "MIT",
  "dependencies": {
    "express": "^4.19.2"
  }
}

apps/orders/server.js

const express = require('express');
const app = express();
const PORT = process.env.PORT || 4000;

app.get('/healthz', (req, res) => res.send('ok'));
app.get('*', (req, res) => {
  res.json({
    service: 'orders',
    path: req.path,
    headers: req.headers
  });
});

app.listen(PORT, () => console.log(`orders service on :${PORT}`));

apps/orders/Dockerfile

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm install --production || npm install
COPY server.js ./
EXPOSE 4000
CMD ["node","server.js"]

---

2) Helm chart for the apps

helm/poc-apps/Chart.yaml

apiVersion: v2
name: poc-apps
description: Two tiny APIs with Ingress (TLS+mTLS+WAF+rate limit)
type: application
version: 0.1.0
appVersion: "0.1.0"

helm/poc-apps/values.yaml

namespace: apps
replicaCount: 2

images:
  node:
    repository: poc-node
    tag: "0.1"
    pullPolicy: IfNotPresent
  orders:
    repository: poc-orders
    tag: "0.1"
    pullPolicy: IfNotPresent

service:
  type: ClusterIP
  node:
    port: 80
    targetPort: 3000
  orders:
    port: 80
    targetPort: 4000

ingress:
  enabled: true
  className: nginx
  host: test.com              # we will map this to Minikube IP via /etc/hosts
  tlsSecretName: tls-testcom  # k8s TLS secret with server cert+key
  clientCASecretName: client-ca  # k8s secret with CA cert for client auth
  rateLimitRPS: "5"
  rateLimitBurst: "10"

helm/poc-apps/templates/deployment-node.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: node
  labels: { app: node }
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels: { app: node }
  template:
    metadata:
      labels: { app: node }
    spec:
      containers:
        - name: node
          image: {{ .Values.images.node.repository }}:{{ .Values.images.node.tag }}
          imagePullPolicy: {{ .Values.images.node.pullPolicy }}
          ports:
            - containerPort: 3000
          readinessProbe:
            httpGet: { path: /healthz, port: 3000 }
            initialDelaySeconds: 2
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz, port: 3000 }
            initialDelaySeconds: 5
            periodSeconds: 10

helm/poc-apps/templates/service-node.yaml

apiVersion: v1
kind: Service
metadata:
  name: node-svc
  labels: { app: node }
spec:
  type: {{ .Values.service.type }}
  selector: { app: node }
  ports:
    - name: http
      port: {{ .Values.service.node.port }}
      targetPort: {{ .Values.service.node.targetPort }}

helm/poc-apps/templates/deployment-orders.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
  labels: { app: orders }
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels: { app: orders }
  template:
    metadata:
      labels: { app: orders }
    spec:
      containers:
        - name: orders
          image: {{ .Values.images.orders.repository }}:{{ .Values.images.orders.tag }}
          imagePullPolicy: {{ .Values.images.orders.pullPolicy }}
          ports:
            - containerPort: 4000
          readinessProbe:
            httpGet: { path: /healthz, port: 4000 }
            initialDelaySeconds: 2
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz, port: 4000 }
            initialDelaySeconds: 5
            periodSeconds: 10

helm/poc-apps/templates/service-orders.yaml

apiVersion: v1
kind: Service
metadata:
  name: orders-svc
  labels: { app: orders }
spec:
  type: {{ .Values.service.type }}
  selector: { app: orders }
  ports:
    - name: http
      port: {{ .Values.service.orders.port }}
      targetPort: {{ .Values.service.orders.targetPort }}

helm/poc-apps/templates/ingress.yaml

{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

    # mTLS (client cert) enforcement
    nginx.ingress.kubernetes.io/auth-tls-secret: "{{ .Release.Namespace }}/{{ .Values.ingress.clientCASecretName }}"
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"

    # Rate limiting (requests per second per client IP)
    nginx.ingress.kubernetes.io/limit-rps: "{{ .Values.ingress.rateLimitRPS }}"
    nginx.ingress.kubernetes.io/limit-burst: "{{ .Values.ingress.rateLimitBurst }}"

    # WAF / ModSecurity
    nginx.ingress.kubernetes.io/enable-modsecurity: "true"
    nginx.ingress.kubernetes.io/enable-owasp-core-rules: "true"

spec:
  ingressClassName: {{ .Values.ingress.className }}
  tls:
    - hosts: [ {{ .Values.ingress.host | quote }} ]
      secretName: {{ .Values.ingress.tlsSecretName }}
  rules:
    - host: {{ .Values.ingress.host }}
      http:
        paths:
          - path: /node
            pathType: Prefix
            backend:
              service:
                name: node-svc
                port:
                  number: {{ .Values.service.node.port }}
          - path: /orders
            pathType: Prefix
            backend:
              service:
                name: orders-svc
                port:
                  number: {{ .Values.service.orders.port }}
{{- end }}

Note: We keep the path prefix (no regex/rewrite). The apps will see /node or /orders in the path, which is fine for a POC.

helm/ingress-nginx/ingress-nginx-waf.yaml

To check for bad inputs

controller:
  config:
    enable-modsecurity: "true"
    enable-owasp-core-rules: "true"
    modsecurity-snippet: |
      SecRuleEngine On
      SecRequestBodyAccess On
      SecResponseBodyAccess Off

Test it later by sending bad input

curl -sS -o /dev/null -w "%{http_code}\n" \
  --cert certs/client.crt --key certs/client.key --cacert certs/ca.crt \
  "https://test.com/node?q=%27%20OR%20%271%27=%271"

---

3) Certificate generation (server TLS + client cert + CA)

scripts/gen-certs.sh

#!/usr/bin/env bash
set -euo pipefail

mkdir -p certs && cd certs

# 1) Create a local CA
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 \
  -subj "/CN=Hitesh Local Dev CA" -out ca.crt

# 2) Create server cert for test.com (signed by our CA)
openssl genrsa -out server.key 2048
openssl req -new -key server.key -subj "/CN=test.com" -out server.csr
cat > server.ext <<EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = test.com
EOF
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt -days 365 -sha256 -extfile server.ext

# 3) Create a client certificate (for mTLS)
openssl genrsa -out client.key 2048
openssl req -new -key client.key -subj "/CN=poc-client" -out client.csr
cat > client.ext <<EOF
basicConstraints=CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth
EOF
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt -days 365 -sha256 -extfile client.ext

# Helpful outputs
openssl x509 -in ca.crt -noout -subject -issuer -dates
openssl x509 -in server.crt -noout -subject -issuer -dates
openssl x509 -in client.crt -noout -subject -issuer -dates

echo "\nCerts in ./certs:"
ls -l

---

4) Step-by-step Commands

Assuming you start inside the repo root (helm-k8s-poc/).

A) Start/prepare Minikube

minikube start
# If you previously enabled the Minikube ingress addon, disable it to avoid conflicts
minikube addons disable ingress || true

B) Install NGINX Ingress Controller (Helm) with WAF

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

# Install into its own namespace and enable ModSecurity + OWASP CRS globally
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  -n ingress-nginx --create-namespace \
  -f ./helm/ingress-nginx/ingress-nginx-waf.yaml

# On Minikube, the Ingress controller Service is type LoadBalancer.
# Run the tunnel in a separate terminal to allocate an external IP.
# Leave this running while you test.
minikube tunnel

C) Build images inside Minikube’s Docker daemon

# Point Docker to Minikube's Docker
eval "$(minikube -p minikube docker-env)"

# Build the two app images
docker build -t poc-node:0.1 ./apps/node
docker build -t poc-orders:0.1 ./apps/orders

D) Create namespace, generate certs, create k8s secrets

kubectl create namespace apps || true

# Generate CA, server cert (test.com), and client cert
bash ./scripts/gen-certs.sh

# 1) TLS secret for server termination at Ingress
kubectl -n apps create secret tls tls-testcom \
  --cert=certs/server.crt --key=certs/server.key --dry-run=client -o yaml | kubectl apply -f -

# 2) CA bundle secret for client cert verification (mTLS)
kubectl -n apps create secret generic client-ca \
  --from-file=ca.crt=certs/ca.crt --dry-run=client -o yaml | kubectl apply -f -

E) Deploy the apps and Ingress via Helm

helm upgrade --install poc-apps ./helm/poc-apps -n apps
# and
kubectl -n apps get deploy,svc,ingress

F) Map a domain locally (simulate owning test.com)

# Get the external IP of the Ingress controller LB
kubectl -n ingress-nginx get svc ingress-nginx-controller -o wide
# Example EXTERNAL-IP: 127.0.0.1 (via tunnel) or 10.96.x.x depending on driver

# Map it to test.com in /etc/hosts (use the EXTERNAL-IP you see)
# On macOS/Linux (requires sudo):
MINIKUBE_IP=$(kubectl -n ingress-nginx get svc ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
# then
echo "$MINIKUBE_IP test.com" | sudo tee -a /etc/hosts

Alternative: use a magic DNS like sslip.io or nip.io and set the Ingress host to something like $(minikube ip).sslip.io to avoid editing /etc/hosts. For this POC we stick with test.com.

G) Smoke-test Services directly (no Ingress)

# Port-forward each service
kubectl -n apps port-forward svc/node-svc 8080:80 &
kubectl -n apps port-forward svc/orders-svc 8081:80 &

curl -s http://127.0.0.1:8080/healthz
curl -s http://127.0.0.1:8081/healthz

H) Test Ingress with mTLS + TLS termination

# Without client cert: should fail (400/401)
curl -vk https://test.com/node

# With client cert: should succeed (200)
curl -vk --cert certs/client.crt --key certs/client.key https://test.com/node
curl -vk --cert certs/client.crt --key certs/client.key https://test.com/orders

I) Test rate limiting (set to ~5 rps)

# Try to exceed 5 rps; some requests should get 429
# Using `ab` (ApacheBench) as an example:
ab -n 50 -c 20 -H "Host: test.com" https://test.com/node/

# If ab has trouble with TLS SNI/Host, use curl in a loop with client cert
for i in $(seq 1 30); do
  curl -sk --cert certs/client.crt --key certs/client.key https://test.com/node >/dev/null &
  sleep 0.05
done; wait

J) Test WAF (ModSecurity + OWASP CRS)

# A crude example payload that often triggers CRS
curl -vk --cert certs/client.crt --key certs/client.key \
  "https://test.com/node?q=%27%20OR%20%271%27=%271"
# Expect 403 if a rule fires; tune with modsecurity-snippet if needed.

---

5) Clean up

helm -n apps uninstall poc-apps
kubectl delete ns apps
helm -n ingress-nginx uninstall ingress-nginx
kubectl delete ns ingress-nginx
# Remove /etc/hosts entry manually if added

---