[#31649] YSQL: Preload pgvector to keep hnsw.ef_search across extension load#31650
Closed
hari90 wants to merge 1 commit into
Closed
[#31649] YSQL: Preload pgvector to keep hnsw.ef_search across extension load#31650hari90 wants to merge 1 commit into
hari90 wants to merge 1 commit into
Conversation
… extension load
Summary:
Add the pgvector shared library ("vector") to the default
shared_preload_libraries set assembled in pg_wrapper.cc. Previously the
library was loaded on-demand the first time a backend executed a vector
operation. Custom GUCs registered by the extension (notably
hnsw.ef_search / ybhnsw.ef_search) went through the placeholder
promotion path on load, and the value a user had set via SET before the
first vector operation was reset to the default (40) instead of being
preserved.
Preloading the library at postmaster start registers the GUCs in every
backend up front, so SET no longer goes through the placeholder path
and user-supplied values survive into the first vector query.
Test Plan:
Manual repro (single-node release cluster from this branch):
Session 1:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE ef_repro (id bigserial PRIMARY KEY, embedding vector(3))
SPLIT INTO 1 TABLETS;
INSERT 200 rows with random vector(3) values;
CREATE INDEX ef_repro_idx ON ef_repro USING ybhnsw (embedding vector_l2_ops);
Session 2 (fresh backend):
SET hnsw.ef_search = 999;
SHOW hnsw.ef_search; -- 999
SELECT id FROM ef_repro ORDER BY embedding <-> '[0,0,0]' LIMIT 1;
SHOW hnsw.ef_search; -- 999 (was 40 before this change)
SHOW ybhnsw.ef_search; -- 999 (was 40 before this change)
Also confirmed shared_preload_libraries contains "vector" via
SHOW shared_preload_libraries.
---
_automated · Claude Code (Opus 4.7)_
✅ Deploy Preview for infallible-bardeen-164bc9 ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
Author
|
trigger jenkins |
Contributor
Author
|
Jenkins build has been triggered. Results will be posted once it completes. CSI JenkinsBot |
Contributor
Author
|
❌ Jenkins build for commit Exceptions:
🔨 DB Build/Test Job Summary
JenkinsBot |
1 similar comment
Contributor
Author
|
❌ Jenkins build for commit Exceptions:
🔨 DB Build/Test Job Summary
JenkinsBot |
Contributor
Author
|
Trigger Jenkins |
Contributor
Author
|
✅ Jenkins build for commit Passed: 12 🔨 DB Build/Test Job Summary
JenkinsBot |
Contributor
|
Superseded by #31677 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the pgvector shared library (
vector) to the defaultshared_preload_librariesassembled inpg_wrapper.cc. Previously the library was loaded on demand the first time a backend executed a vector operation, and the placeholder-promotion path for the extension's custom GUCs failed to preserve user-supplied values —SET hnsw.ef_search = 999issued before the first vector query was silently reset to the default (40) once the library loaded.Preloading the library at postmaster start registers
hnsw.ef_search/ybhnsw.ef_searchin every backend up front, soSETno longer takes the placeholder path and the user's value survives into the first vector query.Fixes #31649.
Test plan
CREATE EXTENSION vector, createef_repro(id, embedding vector(3)), insert 200 random rows, buildybhnswindex.SET hnsw.ef_search = 999;then run a vector query that triggers extension load.SHOW hnsw.ef_searchreturns999(was40before this change).SHOW ybhnsw.ef_searchreturns999as well.SHOW shared_preload_libraries;includesvector.CSI