refactor: optimize table metadata parsing in PostgresIntrospector#1776
Open
rubenferreira97 wants to merge 1 commit intokysely-org:nextfrom
Open
refactor: optimize table metadata parsing in PostgresIntrospector#1776rubenferreira97 wants to merge 1 commit intokysely-org:nextfrom
rubenferreira97 wants to merge 1 commit intokysely-org:nextfrom
Conversation
|
@rubenferreira97 is attempting to deploy a commit to the Kysely Team Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation / Context
This PR is a follow-up optimization to the O(n) improvements made in #1774 for the Postgres introspector bottleneck reported in #1707.
While the previous implementation successfully reduced the time complexity to O(n) using a single
Map, profiling showed that allocating composite string keys (`schema:${schema};table:${table}`) on every iteration created unnecessary overhead for databases with tens of thousands of columns. Furthermore, callingArray.from(tableDictionary.values())at the end added an extra iteration step.Changes
Mapwith a nested structure:Map<string, Map<string, TableMetadata>>(Schema -> Table -> Metadata).tablesarray to push newly created references directly, removing the need forArray.from()at the end of the execution.Performance Impact
I've manually benchmarked these changes locally to confirm the performance gains. To keep the PR clean and avoid polluting the repository with one-off testing files, I haven't included the benchmark scripts in the code changes.
This approach maintains the O(n) lookup complexity but significantly lowers the constant factor overhead by avoiding repetitive string instantiation. Based on previous benchmarks for large schemas (~50,000+ columns), this nested map approach benchmarks roughly ~5x faster (in node) than the string-key lookup method.