feat: saturn query pipeline for rag optimisation

[yekkhan-liftoff]

Summary

Enhanced RAG pipeline with query enhancement and voyage embedding. Query enhancement happens at Slack client layer for all queries, embeddings generated via Voyage AI, and S3 vectors used for similarity search.

Flow

Stage 1: Query Enhancement (Slack Layer)

User Query: "What were Q3 2025 revenues for VX in APAC?"
↓
Slack Client → Query Enhancer (Claude Sonnet)
↓
EnhanceQuery(query, today="2025-11-03")
↓
Analyze: temporal query detected
↓
Output: {
enhancedQuery: "Q3 2025 VX revenues APAC",
metadata: { generatedDate: "2025-09-30" }
}
↓
Date Range Expanded: 2025-09-23 to 2025-09-30 (7-day window)

Stage 2: LLM Tool Detection

Slack Client → Main LLM (GPT-4.1)
↓
CallLLM(enhancedQuery, systemPrompt)
↓
LLM detects: needs rag_search tool
↓
Output: ToolCall {
name: "rag_search",
args: { query: "Q3 2025 VX revenues APAC" }
}

Stage 3: RAG Search with Embeddings

Slack Client → LLM-MCP Bridge
↓
ProcessLLMResponse(extraArgs: {metadata})
↓
Bridge → RAG Client
↓
CallTool("rag_search", args + metadata)
↓
RAG extracts metadata from args
↓
Build SearchOptions {
limit: 30,
dateFilter: ["2025-09-23"..."2025-09-30"]
}
↓
RAG Client → Voyage AI Embedding Provider
↓
POST /v1/embeddings
model: voyage-context-3
dimensions: 1024
↓
Output: queryVector [1024 floats]
↓
RAG Client → AWS S3 Vectors
↓
QueryVectors {
vector: queryVector,
topK: 7,
filter: { report_generated_date: {$in: dates} }
}
↓
S3: Cosine similarity search + metadata filtering
↓
Output: 7 results (scored & sorted)
↓
RAG: sortResultsByDate() - newest first
↓
Format: "Found 7 contexts...
--- Context 1 ---
Source: revenue_report.pdf
Date: 2025-09-30
Content: ..."

Stage 4: Final Synthesis

Bridge → Main LLM
↓
Re-prompt {
query: enhancedQuery,
context: RAG results
}
↓
LLM synthesizes answer from context
↓
Output: "Q3 2025 VX revenues in APAC were..."
↓
Bridge → Slack Client → User

[wyangsun]

Could we use Langchain Go's embedding framework, it support Voyage embedding model.
https://github.com/tmc/langchaingo/blob/main/embeddings/voyageai/voyageai.go
https://github.com/tmc/langchaingo/blob/main/embeddings/embedding.go

[yekkhan-liftoff]

Could we use Langchain Go's embedding framework, it support Voyage embedding model. https://github.com/tmc/langchaingo/blob/main/embeddings/voyageai/voyageai.go https://github.com/tmc/langchaingo/blob/main/embeddings/embedding.go

the model that i am using is not supported yet

[yekkhan-liftoff]

todo: inject query enhancement prompt instead of hardcoding in application

[tommynguyen-vungle]

LGTM