Unified Storage Performance Optimizations #1
Conversation
…#97529) * dont lazy init unified storage * Inits index when creating new resource server. Fixes trace propagation by passing span ctx. Update some logging. * Use finer grained cache locking when building indexes to speed things up. Locking the whole function was slowing things down. * formatting * linter fix * go mod * make update-workspace * fix workspaces check error * update dependency owner in mod file * wait 1 second before querying metrics * try with big timeout, see if fixes CI. Wont fail locally. * skips postgres integration test. Only fails in drone. Will fix later. * put delay back to 500 ms
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return nil, err | ||
| } | ||
|
|
||
| b.cacheMu.Lock() | ||
| b.cache[key] = idx | ||
| b.cacheMu.Unlock() |
There was a problem hiding this comment.
In pkg/storage/unified/search/bleve.go lines 88‑139 the cache mutex is now only taken around the final b.cache[key] = idx assignment. Previously the whole BuildIndex method was executed under cacheMu, which meant GetIndex’s read lock blocked while an index was being built. After this change a second request can call getOrCreateIndex, observe that the cache entry is still nil, and start another BuildIndex for the same NamespacedResource while the first build is still creating the Bleve index on disk. Because both builders call bleve.New for the same directory, one of them will fail with “index path already exists” (or worse, corrupt the files), causing search initialization or the user request to fail whenever two callers race to build the same index. Please restore serialization for each build (e.g. hold the lock until the index has been fully constructed or introduce per‑key locking) so concurrent calls cannot stomp on each other.
Useful? React with 👍 / 👎.
|
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 2 weeks if no further activity occurs. Please feel free to give a status update or ping for review. Thank you for your contributions! |
Test 10
Replicated from ai-code-review-evaluation/grafana-cursor#10