Demonstrate NVBench migration with brute-force k-NN - #2371
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Hi @PointKernel. What's the justification for this change? Are you working on improving brute force knn? Not a great idea to cherry pick algorithms from cuVS bench outside of the existing infra, especially not to yet another benchmark harness. If we are going to make such a change, we should do so for all the supported algorithms. Of course, if this is just for your own internal testing then please feel free to disregard. Opening up a PR tells me you probably want this merged, and in that case, we need to have a much larger discussion about this. |
|
@cjnolet Thanks for raising your concern. I am not specifically working on brute force kNN. I noticed this benchmark was already doing its own GPU timing and synchronization, so it seemed like a simple place to show what NVBench could provide. We migrated all the cuDF GPU benchmarks to NVBench and have found it useful for kernel benchmarks. It takes care of timing, synchronization, warmups, repetitions, and parameter handling that would otherwise need to be implemented manually. Since this benchmark already contains much of that boilerplate, it felt like a good contained example for comparing the two approaches. I agree this should not become a one off in cuVS. I opened the PR so we could discuss the idea around a concrete example rather than an abstract proposal. If cuVS finds this direction useful, this could be the starting point for a broader and more consistent effort across the appropriate GPU benchmarks. |
|
Okay that does makes sense. Thank you for the explanation! |
|
Thank you @cjnolet for checking out this diff and thank you @PointKernel for putting it together. cuDF benchmarks were originally written in googlebench. We started migrating to nvbench in 2021 and just finished in 2025, so there is no need to rush adoption all at once. NVbench is the standard for CCCL and a great tool to make GPU benchmarking easier. My favorite feature is that benchmark parameters can be varied at runtime without recompiling anything. I asked @PointKernel to open the PR to start the conversation with cuVS about nvbench. |
|
Thanks guys. I should probably point out, though, that we are getting close to marking our c++ backend as legacy as we move more towards benchmarking the Python backend in cuVS-bench. It's not formal yet, because we are still in the process of adopting and fleshing out the new python backend, but that is the ultimate goal. Does nvbench support any type of Python layer? |
Yes, I believe so. @oleksandr-pavlyk could probably comment more on this. |
There is no restriction on types to use in the benchmark. You can peruse Python examples of benchmarking at the repo: https://github.com/NVIDIA/nvbench/tree/main/python/examples They include benchmarking of numba-cuda kernels, torch operators, cupy functions, cute-dsl operations, and kernels compiled from source using cuda-core. Benchmarking in Python currently expects that work is submitted on the stream supplied in Unlike C++, Python does not currently allow for |
PointKernel
changed the title
4 participants
This PR migrates the standalone brute-force k-NN benchmark from manual timing to NVBench as a contained example for migrating other appropriate cuVS GPU benchmarks.
It demonstrates fetching static NVBench through RAPIDS CMake, using the standard NVBench main, registering typed and value axes, and keeping input allocation, data initialization, and index construction outside the timed region.
The intent is to establish a reference for incrementally migrating other suitable GPU benchmarks, if the cuVS team sees fit, rather than introduce a one-off benchmark path. End-to-end, CPU, multi-GPU, and hybrid workflows should probably remain on the existing cuVS-bench infrastructure.