Skip to content

RI-8171 Create index when no data exists#6195

Open
dantovska wants to merge 28 commits into
mainfrom
fe/feature/RI-8171/create-index-when-no-data-exists
Open

RI-8171 Create index when no data exists#6195
dantovska wants to merge 28 commits into
mainfrom
fe/feature/RI-8171/create-index-when-no-data-exists

Conversation

@dantovska

@dantovska dantovska commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

What

Allows the Vector Search create-index wizard to be used on databases with no Hash/JSON keys:

  • The welcome screen button is renamed to "Create index" and is always enabled; same for the list page's "Use existing data" menu item (the "no keys" disabled guards are removed).
  • When the database has no Hash/JSON keys, the "Select key" browser panel is hidden and the index definition panel switches to manual creation: centered guidance, "+ Add field", editable index name and prefix, and a HASH/JSON key type toggle (nothing else can set the type when no key is selected). The Command view renders (empty) even before the first field is added.
  • Hardens the existing-keys probe (useHasExistingKeys), since the page layout now depends on it:
    • scans with SCAN_COUNT_DEFAULT instead of count: 1, scanThreshold: 1, which gave up after a single scanned key — false "no keys" whenever a non-Hash/JSON key came first in scan order, and key-by-key round trips;
    • aggregates results across all cluster nodes — previously only data[0] was read, so keys living on any other shard made the wizard claim the database was empty;
    • a failed probe falls back to browse mode instead of hiding the browser;
    • an incomplete scan (threshold reached without finding keys) is treated as inconclusive and also falls back to browse mode — manual creation only appears for a confirmed empty scan;
    • runs once per create-index page visit, and only for the browse flow (sample-data and preselected-key URLs never fire the scans; other Search routes no longer consume it).
  • New/changed strings go through i18n (en + bg).

Notes for review: the prefix is optional in manual mode to match the browse flow (an empty prefix indexes all keys of the selected type), and manual creations report the same data_source: existingData telemetry as the browse flow.

Testing

  • Unit: vector-search suites pass (77 suites / 731 tests), including new cases for manual mode (browser hidden, guidance text, create enabled after adding a field, chosen key type in the generated command, empty command view, loader while checking keys) and for the probe (multi-node cluster response, API error fallback, default scan count, silent re-checks).
  • Manual (standalone): on a DB without Hash/JSON keys — Search → Create index → manual mode; add a field, pick key type → Create → FT.CREATE succeeds and lands on the query page. On a DB with hash keys — the key browser flow is unchanged.

Testing the cluster edge case (keys on a non-first shard)

The keys endpoint returns one scan result per cluster node; before this PR the probe read only the first entry, so keys on any other shard showed manual creation instead of the key browser.

  1. Start a 3-master cluster with search modules (single container so the cluster bus works on macOS):
    docker run -d --name redis-cluster-search \
      -p 7101-7103:7101-7103 -p 17101-17103:17101-17103 \
      redis/redis-stack-server:latest bash -c '
        for port in 7101 7102 7103; do
          redis-server --port $port --cluster-enabled yes \
            --cluster-config-file /tmp/nodes-$port.conf \
            --cluster-announce-ip 127.0.0.1 --protected-mode no --daemonize yes \
            --loadmodule /opt/redis-stack/lib/redisearch.so \
            --loadmodule /opt/redis-stack/lib/rejson.so
        done
        sleep 3
        redis-cli -p 7101 --cluster create 127.0.0.1:7101 127.0.0.1:7102 127.0.0.1:7103 \
          --cluster-replicas 0 --cluster-yes
        tail -f /dev/null'
    (The plain redis Docker images ship without the search module — redis/redis-stack-server is required.)
  2. Add 127.0.0.1:7101 to RedisInsight — it auto-discovers as an OSS Cluster.
  3. Seed exactly one hash key; the {N} hashtag controls which shard it lands on:
    docker exec redis-cluster-search redis-cli -c -p 7101 HSET 'user:{1}' name test
  4. Search tab → Create index: the "Select key" browser must appear and list the key. Repeat for other placements (DEL the key, reseed as user:{2}, user:{3}, user:{4}) — every placement must show the browser. Before the fix, keys on non-first response nodes (discovery order, not slot order) produced manual creation instead.
  5. Cleanup: docker rm -f redis-cluster-search.

Closes #RI-8171

🤖 Generated with Claude Code


Note

Medium Risk
Layout and FT.CREATE inputs now depend on a hardened keys scan (cluster/incomplete-scan edge cases); wrong probe results could show manual vs browse incorrectly, but failures intentionally fall back to browse mode.

Overview
Enables Vector Search index creation when a database has no Hash/JSON keys, instead of blocking entry with “no keys” tooltips and disabled actions.

Entry points: The welcome secondary action and list Use existing data are always available and relabeled to Create index; global hasExistingKeys gating is removed from VectorSearchProvider, welcome screen, and create menu.

Create-index page: On browse (?mode=existingData without a preselected key), useHasExistingKeys runs once (with a loader). A confirmed empty scan switches to manual creation: hide the key browser, show manual empty-state copy, allow + Add field, index name/prefix editing, and a HASH/JSON key-type toggle; create is enabled with at least one field (no key selection). Probe errors or inconclusive scans (incomplete cursor) keep browse mode with the browser visible.

useHasExistingKeys: Scans use SCAN_COUNT_DEFAULT, aggregate all cluster nodes, treat non-zero cursor as “has keys,” expose error and an enabled flag; the hook is no longer used app-wide for disabling UI.

i18n updates in en and bg; unit/e2e tests cover manual mode and probe behavior.

Reviewed by Cursor Bugbot for commit 96f8b9d. Bugbot is set up for automated code reviews on this repo. Configure here.

@dantovska dantovska requested a review from a team as a code owner July 13, 2026 11:35
@dantovska dantovska self-assigned this Jul 13, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e1813fa1a5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Code Coverage - Frontend unit tests

St.
Category Percentage Covered / Total
🟢 Statements 84.03% 27249/32428
🟡 Branches 70.06% 11653/16632
🟡 Functions 79.43% 7302/9193
🟢 Lines 84.5% 26534/31402

Test suite run success

7769 tests passing in 860 suites.

Report generated by 🧪jest coverage report action from 96f8b9d

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4336a2807

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c46018a60

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 44b7cd2d09

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 63150970f0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

dantovska and others added 15 commits July 14, 2026 07:42
Renames the welcome screen "Use data from my database" button to
"Create index" and removes the guard that disabled it when the
database has no Hash/JSON keys. When no keys exist, the create-index
page hides the key browser panel and switches to manual creation:
an empty index definition panel with guidance, where the user adds
fields, an index name, and a prefix themselves.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…x flow

The existing-keys probe scanned with scanThreshold: 1, so it gave up
after the first scanned key — reporting "no keys" whenever a non-Hash/
JSON key happened to come first in scan order. That wrongly sent users
with data into manual creation and kept the list-page "Use existing
data" menu item disabled. Scan with the user-configured threshold
instead, and drop the now-obsolete disabled guard from the menu.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mode

Manual creation required a non-empty index prefix, but the only
feedback was a hover tooltip on the disabled button — adding a field
appeared to do nothing. Drop the hard requirement to match the browse
flow, where an empty prefix (index all keys) is already allowed.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ys probe

Drop the client-supplied scanThreshold — the keys endpoint already caps
scanning with its configured default. Scan with SCAN_COUNT_DEFAULT
instead of count: 1, since the backend derives the per-iteration SCAN
COUNT from it; probing key-by-key could take thousands of round trips
on databases with no Hash/JSON keys. Also replace the `;(... as
jest.Mock)` cast with jest.mocked() in the create-index page spec.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ation

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A failed or partially failed existing-keys probe reported hasKeys=false,
which the create-index page treated as a confirmed empty database and
hid the key browser. Expose an error flag from the probe and only enter
manual creation on a successful "no keys" result.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The existing-keys probe re-runs on route changes and flipped loading
back to true, which unmounted the create-index page (and any form
state) behind a full-page loader. Report loading only for the initial
check; re-checks refresh the result silently.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With no key to derive the type from, manual creation was locked to
FT.CREATE ... ON HASH. Add a HASH/JSON toggle to the toolbar so JSON
indexes can be pre-created on an empty database.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
In manual mode with no fields the content area always shows the
guidance panel, so switching between Table and Command view had no
visible effect. Disable the toggle until the first field is added.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Instead of disabling the view toggle before any fields are added,
render the (empty) command view when the Command tab is selected.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The keys endpoint returns one scan result per cluster node, but the
existing-keys probe only inspected the first entry. Keys living on any
other shard made the create-index page claim the database is empty and
enter manual creation. Aggregate the check across all returned nodes.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The probe cleared its error flag at the start of every re-check, so
after a failed check the create-index page briefly saw error=false with
hasKeys=false and entered manual mode, defeating the browse-mode
fallback. Keep the previous result until the new one lands.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dantovska and others added 5 commits July 14, 2026 07:42
…te-index page

Only the create-index page consumes the probe since the entry-point
guards were removed, yet it still ran from VectorSearchProvider on
every Vector Search route change — two broad key scans per navigation.
Move the hook into the page and drop the now-unused context fields.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…er mount

The probe is page-scoped now, so the silent re-check machinery guarded
transitions that always coincide with an unmount. Every check reports
loading again (a re-check implies a new database context) and the dead
pathname trigger is dropped, so no consumer can observe a stale result.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A scan that stops at the backend threshold without finding Hash/JSON
keys does not prove the database is empty — large mixed keyspaces could
be forced into manual creation while selectable keys exist beyond the
scan window. Enter manual mode only when the scan completed (cursor 0
on every node) and found nothing.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… flow

Sample-data and preselected-key create URLs never use the probe result
but still fired the two Hash/JSON key scans. Gate the hook behind an
enabled flag driven by the browse flow.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A dependency-triggered re-probe could either flash the full-page loader
over an in-progress definition or briefly show a stale result, yet its
only triggers (database or encoding change) always remount the page.
Start the check once per mount and keep the abort tied to unmount, so
neither failure mode is possible.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dantovska dantovska force-pushed the fe/feature/RI-8171/create-index-when-no-data-exists branch from 6315097 to 04a5dcd Compare July 14, 2026 04:54
The once-per-mount guard also suppressed the corrective scan when the
connected instance settles after a database switch, so the layout could
reflect the previous database's keyspace. Key the guard by database and
encoding: exactly one scan per combination, with a stale in-flight scan
aborted when they change.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eeba2f78a3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

dantovska and others added 2 commits July 14, 2026 08:10
With the instance guard moved to the effect, a missing connected
instance id skipped the scan without ever clearing the initial loading
state, leaving the create-index page on the full-page loader. Report an
inconclusive check instead, which falls back to browse mode; the probe
still re-arms once the id settles.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The hook read the connected-instance selector, which can lag behind the
route on a direct create-index navigation between databases — a fast
probe against the previous database could drive the layout for the new
one. Take the database id from the caller (the route param) instead.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dantovska and others added 2 commits July 14, 2026 10:13
…rovider

The page passed both showBrowser and isManualCreation, though one is a
function of the other plus props the provider already receives. Derive
it from mode, preselected key, and isManualCreation instead.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ref-based scan guard and AbortController plumbing duplicated what
effect dependencies already provide: one scan per (enabled, database,
encoding), cancelled on change or unmount. Inline the fetch into the
effect with a cancelled flag. Behavior is unchanged and covered by the
existing tests.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb1478c018

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…r the original

Restore the original hook structure (AbortController, signal handling,
connected-instance selector) and keep only the scan fixes on top of it:
batched scan count, all cluster nodes checked, and an incomplete scan
counting as having keys. Drops the error fallback, enabled gating and
route-id parameter added during review iterations.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the shared existing-data + no-keys setup into a nested beforeEach
and extract an addField helper for the repeated field-modal flow. No
change in coverage.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ea9df719c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…robe

Re-add two guards to the existing-keys probe: a failed or inconclusive
scan now reports an error so the create-index page keeps the key
browser instead of hiding it behind manual creation, and the probe is
gated on the browse flow so sample-data and preselected-key URLs no
longer fire the scans.

References: #RI-8171

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 96f8b9d. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant