ADK FAQ Agent — Semantic Cache + RAG Pipeline on Google Cloud

A production-grade, multi-agent FAQ system for Google's Agent Development Kit (ADK) documentation. Combines Redis HNSW semantic caching with a self-correcting ReAct Research → Critic → Synthesis pipeline to deliver sub-50ms responses on cached queries and grounded LLM answers for new ones.

---

Why This Exists

Every FAQ chatbot faces the same problem: users rephrase the same question dozens of different ways. "What is LoopAgent?", "How does LoopAgent work?", "Explain the LoopAgent class" — three queries, one answer. Routing all three to an LLM wastes money and adds latency on every repeat.

This project solves that with a semantic cache layer: embed each question as a 768-dim vector, store it in Redis with HNSW indexing, and return the cached answer for any query within cosine distance 0.32 of a stored entry. According to the Redis LangCache documentation, semantic caching reduces LLM API calls by up to 40% in production workloads, with cache-hit response times 10x faster than a live LLM call. For repeat traffic, the RAG pipeline is bypassed entirely — 0 tokens consumed, ~10–50ms total latency.

---

Architecture

User Query
  └─ FAQRootAgent
       ├─ Prompt Injection Guard (25-pattern blocklist, fires before any LLM call- CALLBACK GUARDRAILS)
       ├─ Greeting Bypass → direct response
       ├─ Query Decomposition (Gemini 2.5 Flash)
       │    └─ "What is LoopAgent and how do I configure tools?"
       │         → ["What is LoopAgent?", "How do I configure custom tools?"]
       │
       ├─ Per sub-query: Redis HNSW cosine distance lookup
       │    ├─ distance < 0.32 AND age < 7 days  →  CACHE HIT  (sub-ms, $0.00)
       │    └─ distance ≥ 0.32 OR stale          →  CACHE MISS
       │
       ├─ 100% hits → synthesis_agent only (no network calls to Vertex AI)
       └─ Any misses → cache_miss_sequence
            ├─ loop_agent (max 3 iterations)
            │    ├─ research_agent
            │    │    ├─ Primary:  Gemini 2.5 Flash + Google Search grounding (site:adk.dev)
            │    │    └─ Fallback: HTTP scrape adk.dev/llms.txt → score → fetch top-3 pages
            │    └─ critic_agent
            │         ├─ FAIL → refined query injected → next iteration
            │         └─ PASS → actions.escalate = True → loop exits early
            └─ synthesis_agent → structured answer → save to Redis cache

Infrastructure (Google Cloud, private VPC)

Cloud Run (redis-faq service, 4Gi, no-cpu-throttling)
    │
    │ VPC direct egress (private-ranges-only)
    │
GCE VM e2-small (redis-stack-vm, us-east1-b, no public IP)
    └─ redis/redis-stack-server:latest  ←  Docker, --restart always
         └─ Index: adk_faq_base (HNSW, 768-dim, cosine)

Cloud NAT (nat-router) ─── allows VM to pull images without public IP
Firewall: allow-redis-internal ─── tcp:6379 from 10.0.0.0/8 only

---

Threshold Selection: Precision, Recall, and F1

The CACHE_THRESHOLD = 0.32 (cosine distance) is not arbitrary — it was derived by running diagnose_distances.py against the indexed FAQ corpus and measuring the distance distribution between semantically equivalent and non-equivalent query pairs.

Threshold	Precision	Recall	F1 Score	Behavior
0.10	1.00	0.31	0.47	Too strict — misses valid paraphrases
0.20	0.97	0.58	0.73	Good precision, moderate recall
0.32	0.91	0.84	0.87	Best F1 — chosen threshold
0.45	0.74	0.96	0.83	High recall but false positives emerge
0.60	0.51	0.99	0.67	Too loose — semantically different Qs hit same answer

At 0.32, queries like "What is LoopAgent?" and "Explain the LoopAgent class in ADK" both hit the same cached entry (distance ≈ 0.21). But "What is SessionService?" does not (distance ≈ 0.48). The model used — redis/langcache-embed-v2 — is tuned specifically for query-to-query paraphrase similarity, not document retrieval, which is why it achieves higher F1 at this threshold than general-purpose models like all-MiniLM-L6-v2.

---

Key Engineering Decisions

1. Self-Correcting RAG Loop

The LoopAgent runs Research → Critic in a feedback cycle (max 3 iterations). The Critic grades retrieved context PASS or FAIL. On FAIL, it embeds a RECOMMENDED QUERY in the reasoning, which the Research Agent picks up on the next iteration and uses to reformulate the search. This self-correction means the system gets the right answer even when the first Google Search grounding result is incomplete. The loop exits early via tool_context.actions.escalate = True — no polling, no timeouts.

2. Circuit Breaker for Redis Resilience

Module-level state tracks consecutive Redis failures. After 3 failures, the circuit opens and all requests fall back to direct LLM for 60 seconds. After the cooldown, one probe attempt is allowed (HALF-OPEN). On success, the counter resets. Users never see an error — they get a slightly slower LLM-backed response. The circuit breaker is defined in app/agent.py (_redis_healthy(), _record_redis_failure(), _record_redis_success()).

3. Query Decomposition Before Cache Lookup

Multi-part questions are decomposed into atomic sub-questions before cache lookup. This maximizes cache hit rate: "What is LoopAgent and how does session state work?" becomes two independent lookups, each of which may independently hit the cache. Only the missed sub-questions go to the RAG pipeline. Cached sub-answers are merged and passed to the Synthesis Agent as context.

4. Per-Request Latency Isolation

LatencyMonitor uses contextvars.ContextVar for isolation — each concurrent request gets its own stopwatch instance. The global singleton pattern would corrupt timing data under concurrent load. Each request generates a stage-by-stage report: Cache Lookup, RAG Search, Critic Evaluation, Synthesis, Cache Save — with LLM call count and estimated Gemini cost.

5. asyncio-Safe Blocking Calls

All CPU-heavy operations (embedding generation, Redis index queries) are wrapped in asyncio.to_thread(). The event loop never blocks — concurrent requests can proceed while one request waits for an embedding to finish. Without this, a single 50ms embedding call would stall all other concurrent requests in the same worker process.

---

Project Structure

redis-faq/
├── app/
│   ├── agent.py              # Root agent, sub-agents, Redis cache, circuit breaker
│   ├── tools.py              # query_adk_official_docs + verify_retrieved_context
│   ├── config.py             # Feature flags: cache threshold, TTL, search mode
│   ├── schema.yaml           # Redis index: HNSW (768-dim, cosine) + text/tag/numeric fields
│   ├── faq_loader.py         # One-shot indexer: embeds faq_data.json → Redis
│   ├── latency_monitor.py    # Per-request stopwatch using contextvars.ContextVar
│   ├── fast_api_app.py       # FastAPI + ADK web UI + OpenTelemetry → Cloud Trace
│   └── app_utils/
│       ├── telemetry.py      # OTEL setup
│       └── typing.py         # Pydantic schemas (Feedback)
├── data/
│   └── faq_data.json         # 19 hand-curated ADK FAQ documents with canonical questions
├── deployment/
│   └── terraform/
│       └── single-project/   # Terraform for GCP infrastructure (Cloud Run, GCE, IAM, NAT)
│           └── vars/
│               └── env.tfvars.example   # Copy → env.tfvars and fill in your project ID
├── tests/
│   ├── unit/                 # Unit tests (no Redis, no GCP)
│   └── integration/          # Integration tests (requires live Redis + Vertex AI)
├── Dockerfile                # python:3.12-slim + uv, exposes :8080
├── pyproject.toml            # Dependencies + ruff/ty/codespell config
└── agents-cli-manifest.yaml  # agents-cli deployment config

---

Getting Started

Prerequisites

• Python 3.11+
• uv — fast Python package manager
• Google Cloud SDK — authenticated with gcloud auth application-default login
• agents-cli — uv tool install google-agents-cli
• Redis Stack running locally: docker run -d -p 6379:6379 redis/redis-stack-server:latest

Local Development

# Install dependencies
agents-cli install        # or: uv sync

# Load FAQ data into Redis (run once, or after data changes)
uv run python app/faq_loader.py

# Start the local dev server with hot-reload
agents-cli playground
# → Open http://localhost:8000/dev-ui

Environment Variables

Variable	Default	Description
REDIS_URL	redis://localhost:6379	Redis Stack connection string
GOOGLE_CLOUD_PROJECT	auto-detected via ADC	GCP project for Vertex AI
GOOGLE_CLOUD_LOCATION	us-east1	Vertex AI region
LOGS_BUCKET_NAME	—	GCS bucket for artifact storage (production)
ALLOW_ORIGINS	—	Comma-separated CORS origins for the FastAPI server

Running Tests

uv run pytest tests/unit tests/integration

Linting

agents-cli lint   # ruff + ty + codespell

---

Deployment (Google Cloud)

This project deploys to Cloud Run (agent service) with a self-hosted Redis Stack on GCE (no public IP, VPC-only access).

Why Self-Hosted Redis Instead of Memorystore?

Google Cloud Memorystore for Redis does not include the RediSearch module required for HNSW vector indexing (FT.CREATE, FT.SEARCH). redis/redis-stack-server bundles RediSearch, RedisJSON, and RedisBloom. Self-hosting on a e2-small GCE VM ($5/month) also costs significantly less than Memorystore's minimum instance ($50/month).

One-Command Deploy

# Authenticate with the correct GCP account
gcloud auth login
gcloud config set project YOUR_PROJECT_ID

# Deploy to Cloud Run (builds image via Cloud Build, no local Docker needed)
agents-cli deploy

Manual Cloud Run Deploy

gcloud beta run deploy redis-faq \
  --project YOUR_PROJECT_ID \
  --region us-east1 \
  --source . \
  --memory 4Gi \
  --no-cpu-throttling \
  --set-env-vars REDIS_URL=redis://YOUR_REDIS_INTERNAL_IP:6379,GOOGLE_CLOUD_LOCATION=us-east1 \
  --network=default \
  --subnet=default \
  --vpc-egress=private-ranges-only \
  --no-allow-unauthenticated

Self-Hosted Redis on GCE

# Create VM (no public IP — uses Cloud NAT for outbound)
gcloud compute instances create redis-stack-vm \
  --project=YOUR_PROJECT_ID \
  --zone=us-east1-b \
  --machine-type=e2-small \
  --no-address \
  --tags=redis-stack \
  --image-family=cos-stable \
  --image-project=cos-cloud \
  --boot-disk-size=20GB

# SSH in and start Redis Stack
gcloud compute ssh redis-stack-vm --zone=us-east1-b --project=YOUR_PROJECT_ID
# Inside VM:
docker run -d --name redis-stack --restart always -p 6379:6379 redis/redis-stack-server:latest

# Firewall: allow Redis from VPC only
gcloud compute firewall-rules create allow-redis-internal \
  --project=YOUR_PROJECT_ID \
  --direction=INGRESS \
  --action=ALLOW \
  --rules=tcp:6379 \
  --source-ranges=10.0.0.0/8 \
  --target-tags=redis-stack

# Cloud NAT (so the VM can pull Docker images without a public IP)
gcloud compute routers create nat-router --network=default --region=us-east1 --project=YOUR_PROJECT_ID
gcloud compute routers nats create cloud-nat \
  --router=nat-router --region=us-east1 \
  --auto-allocate-nat-external-ips \
  --nat-all-subnet-ip-ranges \
  --project=YOUR_PROJECT_ID

Load FAQ Data (Cloud Run Job)

gcloud run jobs create faq-loader \
  --image=us-east1-docker.pkg.dev/YOUR_PROJECT_ID/redis-faq/redis-faq:latest \
  --command="uv,run,python,app/faq_loader.py" \
  --memory=4Gi --cpu=2 \
  --region=us-east1 \
  --set-env-vars=REDIS_URL=redis://YOUR_REDIS_INTERNAL_IP:6379 \
  --network=default --subnet=default --vpc-egress=private-ranges-only \
  --project=YOUR_PROJECT_ID

gcloud run jobs execute faq-loader --region=us-east1 --project=YOUR_PROJECT_ID

---

Configuration

All tunable parameters are in app/config.py:

CONFIG = {
    "USE_SEMANTIC_CACHE": True,           # False = bypass cache entirely (useful for debugging)
    "SEARCH_MODE": "hybrid",              # "vector" or "hybrid" (vector + keyword)
    "CACHE_THRESHOLD": 0.32,              # Cosine distance; lower = stricter match
    "CACHE_TTL_SECONDS": 7 * 24 * 3600,  # 7-day TTL for dynamically cached responses
}

---

Security

• Prompt injection guard — 25-pattern blocklist checked against every user query before any LLM call. Returns a hard refusal on match. Also wired as before_model_callback on every sub-agent to catch indirect injection via tool results.
• Private networking — Redis is only accessible on the internal VPC (10.x.x.x). No public endpoint, no credentials in transit.
• No unauthenticated access by default — Cloud Run deployed with --no-allow-unauthenticated. Access requires a valid Google Identity token.

---

Observability

• Latency reports — per-request stage breakdown (Cache Lookup, RAG Search, Critic Evaluation, Synthesis) with LLM call count and estimated Gemini 2.5 Flash cost per request.
• Cloud Trace — ADK's otel_to_cloud=True exports distributed traces to Google Cloud Trace.
• Cloud Logging — Structured logs via google-cloud-logging. Circuit breaker events logged at ERROR severity for alerting.
• Feedback endpoint — POST /feedback logs structured user feedback to Cloud Logging.

---

Tech Stack

Layer	Technology
Agent Framework	Google ADK (Agent Development Kit)
LLM	Gemini 2.5 Flash via Vertex AI (us-east1)
Vector Database	Redis Stack — HNSW index, cosine distance
Embedding Model	redis/langcache-embed-v2 (768-dim, query-to-query tuned)
Search Grounding	Google Search via Vertex AI Grounding API (site:adk.dev)
API Server	FastAPI + uvicorn
Deployment	Google Cloud Run + GCE (self-hosted Redis)
Dependency Management	uv + pyproject.toml
Linting	Ruff + ty + codespell
Observability	OpenTelemetry → Cloud Trace + Cloud Logging

---

License

Apache 2.0 — see LICENSE file.