feat(fts): add configurable posting block size

[BubbleCal]

Feature

Linear: OSS-1344

What is the new feature?

FTS inverted index creation now accepts a block_size parameter for compressed posting blocks. Supported values are 128 and 256.

Why do we need this feature?

The posting block size was previously fixed at 128, which made the block-max granularity impossible to tune for different datasets and query profiles.

How does it work?

• Adds block_size to InvertedIndexParams, protobuf details, posting-list schema metadata, and cache headers.
• Uses 256 as the default for newly created indexes.
• Treats older serialized params, schema metadata, and cache entries that omit block_size as legacy 128.
• Rejects unsupported values, including 512, with a clear validation error.
• Threads the configured block size through FTS build, read, iterator, WAND, cache, and MemWAL flush paths.
• Exposes the parameter in Python and Java FTS index creation APIs, with docs and focused tests.

Validation

• cargo fmt --all
• cargo fmt --all --check
• git diff --check
• CARGO_TARGET_DIR=/tmp/lance-target-a479-no512 cargo test -p lance-index block_size -- --nocapture
• CARGO_TARGET_DIR=/tmp/lance-target-a479-no512 cargo clippy -p lance-index --tests -- -D warnings
• uv run make build from python/
• uv run pytest python/tests/test_scalar_index.py::test_create_scalar_index_fts_block_size from python/
• uv run ruff format --check python/tests/test_scalar_index.py python/lance/dataset.py from python/
• uv run ruff check python/tests/test_scalar_index.py python/lance/dataset.py from python/

Not run locally: Java focused test / spotless check, because this machine has no Java Runtime installed (Unable to locate a Java Runtime).

[github-actions]

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

• Start a vote following the Lance community voting process.
  Format specification modifications need 3 binding +1 votes (excluding the
  proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
• Once the vote passes, link the completed vote in this PR. It should not be
  merged until the vote is linked.

[codecov]

Codecov Report

❌ Patch coverage is 93.59862% with 37 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-index/src/scalar/inverted/encoding.rs	91.07%	3 Missing and 7 partials ⚠️
rust/lance-index/src/scalar/inverted/index.rs	95.87%	5 Missing and 3 partials ⚠️
rust/lance-index/src/scalar/inverted/tokenizer.rs	91.66%	3 Missing and 5 partials ⚠️
rust/lance-index/src/scalar/inverted/builder.rs	95.65%	4 Missing ⚠️
...ust/lance-index/src/scalar/inverted/cache_codec.rs	80.00%	1 Missing and 1 partial ⚠️
rust/lance-index/src/scalar/inverted/wand.rs	93.33%	2 Missing ⚠️
rust/lance/src/dataset/mem_wal/index/fts.rs	81.81%	2 Missing ⚠️
rust/lance-index/src/scalar/inverted.rs	96.29%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!