Problem
Retrieval is hardwired to PostgreSQL with pgvector. A deployment that already operates a dedicated search cluster, or that wants to back knowledge search with an external retrieval engine, cannot do so without replacing the query path.
Example
An organization already runs a search cluster with its own operational tooling, scaling story, and on-call rotation. They want Archestra as the agent runtime and their existing cluster as the index, rather than operating a second retrieval system inside their database.
Where
platform/backend/src/knowledge-base/query.ts — queryService, the natural seamplatform/backend/src/models/kb-chunk.ts — vectorSearch, fullTextSearch, findNeighborsplatform/backend/src/knowledge-base/embedder.ts — the write side
Approach
Define an interface behind the query service covering write (index a chunk), read (vector search, keyword search, fetch neighbours), and delete. Keep the PostgreSQL implementation as the default.
Three constraints that make this harder than it first looks:
- ACL filtering must stay ours. Access control cannot be delegated to an external index that does not model our ACL semantics. Either the filter is pushed down and verified, or results come back and are filtered here, which changes the pagination story.
- Citations and context expansion depend on chunk adjacency. An external index has to preserve document and chunk-index identity, not just return text.
- Two implementations need two sets of tests, and the second one is not exercised by our CI unless something runs it.
Done when
- The PostgreSQL path runs through the interface with no behaviour change.
- A second implementation is possible without modifying callers.
- ACL filtering, citations, and context expansion hold for any implementation.
Notes
Ranked last because it is the largest architectural lift and benefits the fewest deployments. If the BM25 work lands, the main practical reason to want an external index may go away.
Problem
Retrieval is hardwired to PostgreSQL with pgvector. A deployment that already operates a dedicated search cluster, or that wants to back knowledge search with an external retrieval engine, cannot do so without replacing the query path.
Example
An organization already runs a search cluster with its own operational tooling, scaling story, and on-call rotation. They want Archestra as the agent runtime and their existing cluster as the index, rather than operating a second retrieval system inside their database.
Where
platform/backend/src/knowledge-base/query.ts—queryService, the natural seamplatform/backend/src/models/kb-chunk.ts—vectorSearch,fullTextSearch,findNeighborsplatform/backend/src/knowledge-base/embedder.ts— the write sideApproach
Define an interface behind the query service covering write (index a chunk), read (vector search, keyword search, fetch neighbours), and delete. Keep the PostgreSQL implementation as the default.
Three constraints that make this harder than it first looks:
Done when
Notes
Ranked last because it is the largest architectural lift and benefits the fewest deployments. If the BM25 work lands, the main practical reason to want an external index may go away.