From eb0ae5073576dc33251cf9ee7d8f9a8cd5651dae Mon Sep 17 00:00:00 2001 From: Ahmed Metwally Date: Mon, 14 Sep 2026 19:52:48 -0700 Subject: [PATCH 1/2] Document the row / record / TermsAndValues levels Four names sit close enough together to read as duplicates of each other, and the pairing of record with TermsAndValues has now been flagged twice as though one of them were redundant. They are four levels: a row is a record plus metadata, identified by a rowNum; a record is the feature data as a concept, whose shape RecordType names; TermsAndValues is the public array pair a record arrives in; LongTermsAndValues is the internal form with terms hashed to long and the uniValue cached alongside. Stating it also explains why the enum is RecordType rather than TermsAndValuesType. README said "A record is a TermsAndValues", which flattens the two levels into each other. It now says a record reaches USSI as one, which is the same instruction for an embedder without the conflation. Docs only. --- DESIGN.md | 27 +++++++++++++++++++++++++++ README.md | 2 +- 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/DESIGN.md b/DESIGN.md index 0b4659a..2462389 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -7,6 +7,33 @@ library. If you are embedding it, [README.md](README.md) is what you want, and the public `NearestNeighborSearchIndex` API is what you should use rather than managing any of the structures below directly. +## Rows and Records + +Four names sit close enough together to look like duplicates. They are four +levels of one thing. + +A **row** is the unit a searchable structure stores: a record together with +its metadata, the extra data a filter matches on. A row is identified by a +`rowNum`, which is the handle the internals pass around in place of the row +itself. + +A **record** is the feature data of a row. It is a concept, not a type, and +it can take the shape of a vector, a time series, a histogram, a sparse +(multi-)set, or a sequence of terms. `RecordType` names which shape a given +record is in; a comparator declares the shapes it reads and a structure +declares the one it stores, so the two pair up by agreeing on one. + +`TermsAndValues` is the public data structure a record arrives in: two +parallel arrays, `String[] terms` and `float[] values`, which between them +express every shape above depending on how they are populated. Terms alone +is a sequence, values alone is dense, both together is sparse. + +`LongTermsAndValues` is the internal form of the same pair, with terms +hashed to `long` and the comparator-derived `uniValue` cached alongside them. + +So a record is the idea and a `TermsAndValues` is the array pair implementing +it, which is why the enum is `RecordType` and not `TermsAndValuesType`. + ## Structure Lifecycle `NearestNeighborSearchIndex` owns one active cache, zero or more graduating diff --git a/README.md b/README.md index ca214e6..fd53ec5 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ that cannot work together all fail at creation. ## Records -A record is a `TermsAndValues`, built from parallel arrays: +A record reaches USSI as a `TermsAndValues`, built from parallel arrays: ```java new TermsAndValues(String[] terms, float[] values) From eff550b0d3b430fddc50affd4f05ebb88c6f6292 Mon Sep 17 00:00:00 2001 From: Ahmed Metwally Date: Mon, 14 Sep 2026 20:04:17 -0700 Subject: [PATCH 2/2] Document the two senses of "index" The audit flagged NearestNeighborSearchIndex as misnamed on the grounds that it holds List and List, so it is the engine rather than an index. That reasoning missed that the word covers two levels and both are right in their place: the library as a whole is an index, which is the sense Uber Similarity Search Index carries, while an Index is one searchable structure inside it. So the facade keeps its name, and the actual confusion - one word, two levels - is stated where it belongs rather than fixed by a rename that would leave a library called an index with nothing named one at the top. --- DESIGN.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2462389..e9c0095 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -127,6 +127,23 @@ Enums a config names implement `ConfigVocabulary`, which holds the one case-insensitive lookup and the one unsupported-value message. A config never names a record type, so `RecordType` is not among them. +## Two Senses Of "Index" + +The word covers two levels, and both are correct in their place. + +The library is an index: `NearestNeighborSearchIndex` is the whole searchable +thing, which is the sense the project's own name carries in Uber Similarity +Search Index. It stores rows, answers queries, and is the only type an +embedder needs. + +An `Index` is one searchable structure inside it, a sibling of a `Cache` and +a `SearchableStructure` like it. The facade owns a list of these and a list +of caches, and merges their results. + +So the facade holding indexes rather than being one is not a contradiction. +A reader who expects `NearestNeighborSearchIndex` to extend `Index` has the +narrow sense in mind; nothing does, and nothing should. + ## Package Layout ### Searchable Structures