perf(fts): prewarm larger chunks concurrently

[BubbleCal]

Performance Improvement

What is the performance issue or bottleneck?

FTS prewarm was reading posting lists in very small chunks. For large FTS indexes this can create thousands of range reads, so remote-read scheduling and per-chunk build overhead dominate prewarm time.

How does this PR improve performance?

This PR uses larger bounded chunks and bounded read/build concurrency:

• Increase the default prewarm chunk target from 32 MiB to 128 MiB.
• Increase the token cap from 4,096 tokens to 256K tokens.
• Read/build chunks concurrently within the current posting partition, using the store I/O parallelism as the concurrency limit.
• Prewarm posting partitions serially to avoid multiplying partition-level and chunk-level fanout.
• Preserve group-aligned chunk boundaries and with_position=true cache behavior.

This keeps prewarm bounded without using a whole-file fast path or runtime memory-budget probing.

Benchmark or measurement results:

On the large FTS prewarm benchmark used for validation:

• Old main: 434.200s.
• This PR: 35.216s and 35.165s, mean 35.191s.
• About 12.34x faster than old main.
• Prewarm wall time reduced by about 91.9%.

Validation

• cargo fmt --all
• git diff --check
• cargo test -p lance-index prewarm -- --nocapture
• cargo clippy --all --tests --benches -- -D warnings

[codecov]

Codecov Report

❌ Patch coverage is 96.63866% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-index/src/scalar/inverted/index.rs	96.63%	3 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!