feat: add ICU split tokenizer variant

[Xuanwo]

This adds an opt-in icu/split tokenizer variant for FTS so mixed-language text can keep ICU segmentation while also splitting identifier-like tokens on simple-style delimiters. The existing icu tokenizer remains unchanged, and the variant is configured through the existing base_tokenizer surface.

Closes #7280.

Validation note: uv run make lint currently fails at pyright in this local environment because optional tensorflow and torch imports cannot be resolved; ruff formatting and checks passed before that failure.

[github-actions]

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

• Start a vote following the Lance community voting process.
  Format specification modifications need 3 binding +1 votes (excluding the
  proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
• Once the vote passes, link the completed vote in this PR. It should not be
  merged until the vote is linked.

[BubbleCal]

Nice work!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!