Skip to content

Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search #1907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
afourniernv wants to merge 41 commits into
base: main
Choose a base branch
from
Open

Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search #1907

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
336432d
Add cuvs-bench-elastic: HTTP backend for Elasticsearch GPU vector search
afourniernv Mar 10, 2026
80c104f
Fix bulk indexing format for Elasticsearch
afourniernv Mar 20, 2026
7b25521
Add run_build, run_search, run_benchmark API for elastic backend
afourniernv Mar 20, 2026
c224a8c
Update cuvs-bench-elastic README and config loader
afourniernv Mar 20, 2026
e499469
Add cuvs-bench-elastic: Elasticsearch GPU backend plugin
afourniernv Apr 12, 2026
087289d
Revert unrelated change to get_dataset/__main__.py
afourniernv Apr 12, 2026
e8f8a64
Fold Elasticsearch plugin back into cuvs_bench; wire via entry points
afourniernv Apr 13, 2026
b787396
Add run_build/run_search/run_benchmark convenience API to elasticsear…
afourniernv Apr 13, 2026
94a9199
Merge branch 'main' into pr1907-reconcile
afourniernv May 22, 2026
4e35484
Fix elasticsearch backend after cuvs-bench API changes
afourniernv May 20, 2026
e00c288
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv May 22, 2026
1264a98
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jun 5, 2026
cd93845
Improve elasticsearch backend validation
afourniernv Jun 5, 2026
4b0b1f0
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jun 10, 2026
ad40caa
Tighten optional backend handling
afourniernv Jun 11, 2026
1d2a1a9
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jun 11, 2026
680c2ab
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jun 12, 2026
075a8a8
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jul 13, 2026
a0a4b67
Remove unused elastic vector loader wrappers
afourniernv Jul 13, 2026
fe34fd4
Remove redundant elastic config loader override
afourniernv Jul 13, 2026
1b81f44
Use elastic index config as build param source
afourniernv Jul 13, 2026
fcc13f5
Remove elastic benchmark convenience wrappers
afourniernv Jul 13, 2026
cb5b0fd
Fix elastic test registration fallback
afourniernv Jul 13, 2026
8e03291
Align elastic benchmark dependency metadata
afourniernv Jul 13, 2026
da8aafa
Normalize elastic algorithm variants in loader
afourniernv Jul 13, 2026
3f80d6b
Load split plugin entry points across registry groups
afourniernv Jul 13, 2026
e30bdd1
Route elastic recall through orchestrator
afourniernv Jul 13, 2026
df0cb52
Clean up elastic loader wrapper tests
afourniernv Jul 13, 2026
30d85e6
Fix elastic pytest regressions
afourniernv Jul 13, 2026
edae65e
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jul 20, 2026
4df7c9c
Address final elastic review nits
afourniernv Jul 20, 2026
1c1a384
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jul 21, 2026
e5ba1d1
Merge branch 'main' into fea-1856-cuvs-lucene-backend
cjnolet Jul 22, 2026
ba9c112
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jul 24, 2026
3bcdd26
Merge remote-tracking branch 'fork/fea-1856-cuvs-lucene-backend' into…
afourniernv Jul 24, 2026
c2fdf57
Merge branch 'main' into fea-1856-cuvs-lucene-backend
cjnolet Jul 27, 2026
5429b50
Merge branch 'main' into fea-1856-cuvs-lucene-backend
cjnolet Jul 28, 2026
9a065a5
Rename Elasticsearch backend class and config
afourniernv Jul 29, 2026
16060fc
Use yml extension for Elasticsearch config
afourniernv Jul 29, 2026
1305b70
Merge remote-tracking branch 'origin/main' into pr1907-reconcile
afourniernv Jul 29, 2026
8e1492c
Apply pre-commit style fixes
afourniernv Jul 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,14 @@ files:
table: project
includes:
- bench_python
py_elastic_cuvs_bench:
output: pyproject
pyproject_dir: python/cuvs_bench
extras:
table: project.optional-dependencies
key: elastic
includes:
- bench_elastic
channels:
- rapidsai-nightly
- rapidsai
Expand Down Expand Up @@ -500,6 +508,11 @@ dependencies:
- output_types: [requirements, pyproject]
packages:
- matplotlib>=3.9
bench_elastic:
common:
- output_types: [conda, pyproject, requirements]
packages:
- cuvs-bench-elastic>=26.4.0

@coderabbitai coderabbitai Bot Jun 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== pyproject elastic extra ==="
sed -n '41,56p' python/cuvs_bench/pyproject.toml

echo
echo "=== dependencies.yaml elastic source-of-truth ==="
rg -n "py_elastic_cuvs_bench|bench_elastic|cuvs-bench-elastic|elasticsearch>=8.0" dependencies.yaml -A4 -B3

Repository: rapidsai/cuvs

Length of output: 1500

Fix elastic extra source-of-truth drift between dependencies.yaml and generated pyproject.toml

dependencies.yaml maps project.optional-dependencies.elastic (via py_elastic_cuvs_bench → bench_elastic) to cuvs-bench-elastic>=26.4.0, but python/cuvs_bench/pyproject.toml currently declares elastic = ["elasticsearch>=8.0"]—regenerating from dependencies.yaml will overwrite the committed pyproject extra and create install-hint/metadata drift.

Suggested fix 
   bench_elastic:
     common:
       - output_types: [conda, pyproject, requirements]
         packages:
-          - cuvs-bench-elastic>=26.4.0
+          - elasticsearch>=8.0
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dependencies.yaml` around lines 596 - 600, The pyproject extra for "elastic"
is drifting because dependencies.yaml defines bench_elastic ->
cuvs-bench-elastic>=26.4.0 while python/cuvs_bench/pyproject.toml currently
lists elastic = ["elasticsearch>=8.0"]; update the source-of-truth to keep them
in sync by changing dependencies.yaml's bench_elastic entry to include the same
package(s) as the pyproject extra (or update python/cuvs_bench/pyproject.toml to
match dependencies.yaml) so that project.optional-dependencies.elastic (via
py_elastic_cuvs_bench / bench_elastic) consistently maps to the intended package
list and version constraint.

Source: Coding guidelines

@afourniernv afourniernv Jul 13, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8e03291. dependencies.yaml is now the source of truth for the elastic extra again and matches python/cuvs_bench/pyproject.toml by depending on elasticsearch>=8.0 directly, consistent with the in-tree optional-backend model.

depends_on_cuda_python:
specific:
- output_types: [conda, requirements, pyproject]
Expand Down
Loading
Loading