Skip to content

CTX7-1872: Improve Context7 documentation queries#2941

Open
enesgules wants to merge 6 commits into
masterfrom
CTX7-1872/focus-context7-documentation-queries
Open

CTX7-1872: Improve Context7 documentation queries#2941
enesgules wants to merge 6 commits into
masterfrom
CTX7-1872/focus-context7-documentation-queries

Conversation

@enesgules

@enesgules enesgules commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

What changed

Updated MCP, CLI, SDK, Pi, plugin, rule, and skill prompts to ask agents what to look up in a library's documentation instead of passing the user's question or task directly.

Existing guidance remains intact, including examples, single-topic queries, multi-concept splitting, and the exception for questions about how concepts interact. The final find-docs wording distinguishes the documentation lookup from the task the agent needs to complete.

Why

Agents can send broad implementation requests or project context to Context7 even though Context7 retrieves documentation. These prompts keep the query about the documentation needed while preserving useful natural-language documentation questions.

Benchmark

The CLI benchmark used the same 80 questions for production and local targets. In the latest run:

  • local score: 7.09, production score: 7.01
  • local documentation calls: 1.43, production: 1.56
  • local duration: 46.75s, production: 54.47s
  • question-style documentation queries: 7 for both
  • how to documentation queries: 2 local, 4 production
  • action-style implement documentation queries: 0 local, 1 production

Benchmark harness: https://github.com/upstash/context7benchmark/pull/16

Checks

  • MCP typecheck, lint, and 46 tests
  • CLI typecheck, lint, and 245 tests
  • Pi typecheck, lint, and 4 tests
  • AI SDK typecheck and 11 local tests
  • CLI skill-alignment tests, 3 passed
  • git diff --check
  • Production-versus-local CLI benchmark, 80 questions

Linear: https://linear.app/upstash/issue/CTX7-1872/focus-context7-documentation-queries

App prompt: https://github.com/upstash/context7app/pull/814

@linear-code

linear-code Bot commented Jul 23, 2026

Copy link
Copy Markdown

CTX7-1872

@enesgules enesgules changed the title CTX7-1872: Focus Context7 documentation queries CTX7-1872: Improve Context7 documentation queries Jul 24, 2026
@enesgules
enesgules marked this pull request as ready for review July 24, 2026 13:00
@enesgules

Copy link
Copy Markdown
Collaborator Author

CI note: the fresh run and one retry both passed lint, formatting, build, typecheck, 46 MCP tests, and 245 CLI tests. The only failure is the existing live Pi resolve-library-id API test reaching its hard 5-second timeout. This network call does not exercise the prompt wording changed by this PR.

@enesgules

Copy link
Copy Markdown
Collaborator Author

Fixed in aa239bf: the live Pi API test now has a test-local 15-second timeout instead of Vitest’s 5-second default. The full GitHub Actions test workflow is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants