Skip to content

Parallel index build - #251

Merged
syvb merged 42 commits into
mainfrom
sv/parallel-2
Nov 4, 2025
Merged

Parallel index build#251
syvb merged 42 commits into
mainfrom
sv/parallel-2

Conversation

@syvb

@syvb syvb commented Sep 11, 2025

Copy link
Copy Markdown
Contributor

No description provided.

@syvb
syvb marked this pull request as ready for review September 22, 2025 06:15
@syvb
syvb requested a review from a team as a code owner September 22, 2025 06:15
@syvb
syvb requested review from Copilot and tjgreen42 September 22, 2025 06:15

This comment was marked as resolved.

@tjgreen42 tjgreen42 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with initial pass. I didn't see any test-related changes: we need full unit test coverage both with and without parallelism enabled. Hard-coded limits (like # tuples in base relation => parallel vs not) will need parameterization for this purpose, otherwise unit tests will just revert to serial mode.

Comment thread pgvectorscale/src/access_method/build.rs
Comment thread pgvectorscale/src/access_method/build.rs Outdated
Comment thread pgvectorscale/src/access_method/build.rs Outdated
Comment thread pgvectorscale/src/access_method/build.rs Outdated
Comment thread pgvectorscale/src/access_method/build.rs Outdated
Comment thread pgvectorscale/src/access_method/graph/mod.rs Outdated
Comment thread pgvectorscale/src/access_method/graph/neighbor_store.rs Outdated
Comment thread pgvectorscale/src/access_method/graph/neighbor_store.rs Outdated
Comment thread pgvectorscale/src/access_method/graph/neighbor_store.rs Outdated
Comment thread pgvectorscale/src/access_method/meta_page.rs Outdated
@syvb

syvb commented Sep 30, 2025

Copy link
Copy Markdown
Contributor Author

The parallel build logic I implemented doesn't work on PG13 - I've removed the CI build for it in parallel build (because it doesn't compile), and it's going to be EOL in a few months so I think it's fine to just remove PG13 support?

@tjgreen42 tjgreen42 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting close. Besides issues mentioned below, we need some documentation updates to the README as part of this PR.

Comment thread pgvectorscale/src/access_method/build.rs
Comment thread pgvectorscale/src/access_method/build.rs Outdated
@syvb
syvb requested a review from tjgreen42 October 29, 2025 15:34
@tjgreen42

Copy link
Copy Markdown
Collaborator

The parallel build logic I implemented doesn't work on PG13 - I've removed the CI build for it in parallel build (because it doesn't compile), and it's going to be EOL in a few months so I think it's fine to just remove PG13 support?

Did you want to disable PG13 still? We probably should anyway.

Comment thread .github/workflows/pgrx_test.yaml Outdated
Comment thread pgvectorscale/src/access_method/build/parallel.rs Outdated
Comment thread pgvectorscale/src/access_method/build/parallel.rs
Comment thread pgvectorscale/src/access_method/graph/neighbor_store.rs Outdated
Comment thread pgvectorscale/src/access_method/graph/neighbor_store.rs Outdated
Comment thread pgvectorscale/src/access_method/guc.rs Outdated
Comment thread pgvectorscale/src/access_method/guc.rs
Comment thread pgvectorscale/src/access_method/mod.rs Outdated
@syvb
syvb requested a review from tjgreen42 November 3, 2025 07:29
tjgreen42
tjgreen42 previously approved these changes Nov 3, 2025
@syvb
syvb merged commit 7a96623 into main Nov 4, 2025
53 checks passed
@syvb
syvb deleted the sv/parallel-2 branch November 4, 2025 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants