Introduce blog post for disk-based k-NN #3616
Conversation
Adds a blog post for disk-based k-NN. Included is a set of results and images. Signed-off-by: John Mazanec <[email protected]>
jmazanec15
requested review from
elfisher,
AMoo-Miki,
nknize,
krisfreedain,
peterzhuamazon,
CEHENKLE,
dtaivpp,
kolchfa-aws,
nateynateynate and
natebower
as code owners
|
|
||
| | Metric/Configuration | in-memory | on_disk_8x | in_memory_8x | on_disk_16x | in_memory_16x | on_disk_32x | in_memory_32x | | ||
| |-----------------------------------|-----------|------------|--------------|-------------|---------------|-------------|---------------| | ||
| | recall@100 (ratio) | 0.95 | 0.98 | 0.98 | 0.97 | 0.96 | 0.94 | 0.95 | |
There was a problem hiding this comment.
This shows that 32x compression just works and we should not add on_disk here
There was a problem hiding this comment.
Added both just for sake of transparency. For some data sets, the re-scoring does not significantly help.
Co-authored-by: Navneet Verma <[email protected]> Signed-off-by: John Mazanec <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Co-authored-by: John Mazanec <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
There was a problem hiding this comment.
@kolchfa-aws @jmazanec15 Editorial review complete. Please see my comments and changes and let me know if you have any questions. Thanks!
Cc: @pajuric
| Interestingly, for this dataset, the on-disk approach with rescoring produces similar recall to the in-memory approach without rescoring, but the in-memory approach is substantially faster. This is most likely because the Cohere v3 model has been optimized to work very well with binary quantized data (see [this blog post](https://cohere.com/blog/int8-binary-embeddings)). | ||
|
|
||
| ## Learnings | ||
|
|
There was a problem hiding this comment.
Line 260: We haven't actually referenced ANN prior to this. Instead of "ANN approach", do we mean "nearest neighbor approach"?
There was a problem hiding this comment.
Approximate nearest neighbor search
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
|
@pajuric Could you please edit the meta for this blog, and it will be ready to publish. Thanks! |
|
@pajuric @kolchfa-aws Are there any todos on this one - or can we publish? |
|
@jmazanec15 We can publish after @pajuric updates the meta. |
Signed-off-by: John Mazanec <[email protected]>
|
Thanks @pajuric - updated |
|
@nateynateynate @krisfreedain - New blog to publish today |
Description
Adds a blog for disk-based vector search
Issues Resolved
#3615
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.