Repository navigation
fix: retain IVF-RQ batch parallelism and parse release TOML - #125
Conversation
|
Benchmark run · success · attempt 1 Vector index benchmark🟢 No alerts (5/5 indexes)
🟡 QPS ↓ >10% or Recall ↓ >1pp · 🔴 QPS ↓ >20% or Recall ↓ >3pp. Advisory only; QPS is measured after warmup. Absolute metrics & environmentMedians, base → PR. QPS is timed after warmup; P95 uses the first warm sweep.
Base |
leaves12138
left a comment
There was a problem hiding this comment.
LGTM. Reviewed head 281a3566f016726ddfd0c52715cfad09b49eb41a.
The streamed chunk remains shared and read-only while each top-K query retains exclusive access to its heap and statistics. Range-search workers propagate errors through try_for_each before advancing to the next chunk. The release preflight now reads the actual TOML project version; the Python 3.9 CI job installs the backport, and the release-preflight workflow uses Python 3.12 with built-in tomllib.
Independent validation:
cargo test --locked --workspace --no-fail-fast: 738 passed, 2 ignored.- Tool/release unit tests on Python 3.10: 28 passed.
- Formatting, Clippy with warnings denied, source license headers, and binary legal inventory checks passed.
- An additional temporary regression used a 67,714,854-byte list with the default streaming limits. Top-K and range results matched the full-read path; four Rayon workers participated. One-worker and four-worker streaming runs used the same 344 read calls across both searches, with a maximum request of 712,704 bytes.
- An additional temporary regression confirmed that an invalid factor in a later streamed chunk propagates
InvalidDatafrom parallel range search.
No blocking issues found.
(cherry picked from commit b8736bb)
Follow-up to #123.
Oversized IVF-RQ lists changed batch scans from parallel query execution to a single worker. The release preflight could also mistake a version example inside multiline TOML text for the actual Python project version.
Changes
tomllib/tomli, including comments and quoted values, rather than matching text with a regex. Installtomliconditionally for the Python 3.9 CI checks.Validation
cargo test --workspace --no-fail-fast: 738 passed, 2 existing ignored.