fix: harden LLM rerank fallback for reasoning models

[guiyangyuan]

Description

Harden the knowledge search LLM reranker when a reasoning-capable model exhausts its output budget or returns a partial structured score list.

• disable model thinking for the deterministic scoring call and reserve output tokens for providers that ignore the option
• require the response to contain exactly one score per passage instead of padding or truncating partial output
• stop subsequent rerank batches after incomplete output and fall back to the original search scores for all remaining results
• add regression coverage for token exhaustion and incomplete score lists

This avoids spending additional LLM latency after the first unusable batch and preserves the original retrieval ranking instead of manufacturing missing scores.

Type of Change

• 🐛 Bug fix
• ✨ New feature
• 💥 Breaking change
• 📚 Documentation update
• 🎨 Refactor
• ⚡ Performance improvement
• 🧪 Test
• 🔧 Configuration / Build / CI

Related Issue

Partially addresses #2703.

Testing

• go test ./internal/agent/tools -run 'TestRerankChatStubStreamIsClosed|TestRerankWithLLMStopsAfterIncompleteOutput|TestParseScoresFromResponseRequiresExactCount|TestRerankThreshold_default|TestFilterRerankRankResults|TestApplyModelRerankScores' -count=1
• go test -race ./internal/agent/tools -run 'TestRerankChatStubStreamIsClosed|TestRerankWithLLMStopsAfterIncompleteOutput|TestParseScoresFromResponseRequiresExactCount' -count=1
• go vet ./internal/agent/tools
• git diff --check upstream/main...HEAD

The full go test ./internal/agent/tools -count=1 command compiled but did not finish in the local environment after more than two minutes and was interrupted without test failure output. golangci-lint was not available locally.

Checklist

• git diff --check origin/main...HEAD passes
• Changed source files are formatted
• Targeted tests for the changed packages/components pass
• Diff-scoped lint passes where applicable (for Go: golangci-lint run --new-from-rev=origin/main ./...)
• Full-repository checks were run, or any unrelated/environment-dependent failures are documented above
• Self-reviewed the code
• Added/updated tests covering the change
• Updated related documentation (not applicable)
• Breaking changes are clearly called out (none)

Screenshots / Recordings

Not applicable; this is a backend-only change.

[Copilot]

Pull request overview

This PR hardens the knowledge_search LLM-based rerank fallback path to handle reasoning-capable models that may exhaust output budgets or return incomplete score lists, ensuring the system avoids wasted latency and preserves original retrieval ordering when structured rerank output is unusable.

Changes:

• Disables model “thinking” for deterministic rerank scoring calls and increases max_tokens with a fixed reserve.
• Requires an exact score count per batch; on incomplete/invalid output, stops further batches and falls back to original retrieval scores for remaining results.
• Adds regression tests covering token exhaustion (finish_reason=length) and incomplete score lists.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
internal/agent/tools/knowledge_search.go	Tightens LLM rerank output validation, disables reasoning for scoring calls, and short-circuits remaining batches on incomplete output.
internal/agent/tools/knowledge_search_rerank_test.go	Adds regression tests to ensure rerank stops after invalid batches and enforces exact score counts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.