Repository navigation
perf(flat): accelerate scans and cache IVF partitions - #122
Conversation
leaves12138
left a comment
There was a problem hiding this comment.
Reviewed commit f8ffdbb. No blocking issues found.
Checked the partition-cache accounting and eviction, unfiltered cache reuse with query-local filtering, shared batch-filter compaction, the short-vector L2 Top-K path, and indirect-heap replacement/tie ordering. The follow-up test changes address the initial Rust/JNI failures without changing range-search kernel-consumption semantics or relying on payload reads after cache warmup.
Validation on this commit in an isolated checkout:
cargo fmt --all -- --checkcargo clippy --all-targets --workspace -- -D warningscargo test --workspace --no-fail-fastusing a fresh target directory: 723 passed, 2 ignored.- Java Maven test phase and
VectorIndexNativeValidationTest, including the callback-error checks and 168 core range-search oracle cases.
Some GitHub checks are still queued/running at review time; the approval is based on code inspection and the local verification above. The reported large-dataset performance measurements were not independently rerun.
|
Benchmark run · success · attempt 1 Vector index benchmark🟢 No alerts (5/5 indexes)
🟡 QPS ↓ >10% or Recall ↓ >1pp · 🔴 QPS ↓ >20% or Recall ↓ >3pp. Advisory only; QPS is measured after warmup. Absolute metrics & environmentMedians, base → PR. QPS is timed after warmup; P95 uses the first warm sweep.
Base |
IVF-FLAT repeatedly decoded partition payloads across searches, evaluated shared batch filters for each query, and rescanned all Top-K slots after accepted candidates. Reuse aligned, unfiltered partitions within the reader memory budget, compact filters shared by enough queries, batch short-vector L2 scans, and use an indirect max heap for K >= 32 while preserving tie ordering.
The unified reader now passes its existing memory-budget options to IVF-FLAT. Direct
IVFFlatIndexReader::openremains uncached; the newopen_with_optionsenables caching. Oversized lists bypass the cache. Regression tests cover cache bounds, filter isolation, I/O retries, metadata invalidation, heap ties, and range-search semantics. The v1 storage format is unchanged.Local project
ann_benchmeasurements (M4 Pro, 8 threads, Top-10, nprobe=64, two runs per version) showed warm batch QPS gains of 43% on SIFT1M, 4% on GIST1M, and 53% on GloVe100, with unchanged recall. Warm single-query throughput improved 5.15–5.82x. GIST's first batch regressed approximately 16%, so high-dimensional batch gains are limited and cache-fill overhead remains a tradeoff.Validation on an isolated checkout of this commit:
cargo fmt --all -- --checkcargo clippy --all-targets --workspace -- -D warningscargo test --workspace --quiet: 723 passed, 2 ignored.Benchmark harnesses, scripts, and documentation changes are excluded from this PR.