Skip to content

feat(guard): screen Claude Agent SDK tool results before the model reads them - #271

Open
arcjet-rei wants to merge 11 commits into
mainfrom
rei/eng-1536-guard-tool-result-rules
Open

arcjet-rei wants to merge 11 commits into
mainfrom
rei/eng-1536-guard-tool-result-rules

Conversation

@arcjet-rei

@arcjet-rei arcjet-rei commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Adds two ways to screen what a Claude Agent SDK tool returned, before the model reads it:

  • guard_tool(result_rules=, result_action=) for authored tools: a second Guard call over the handler's return value, labelled {action}.result by default. A DENY returns the existing denial envelope in place of the result.
  • guard_hooks(tool_results={...}) for every tool, on PostToolUse. A DENY replaces the result through updatedToolOutput for MCP tools and for the built-ins in REPLACEABLE_BUILTIN_TOOLS (Bash, Read). For any other tool the CLI would reject the replacement and send the original, so the run stops with continue: false instead, and the capture records which happened.

Claude Code validates a built-in tool's replacement against an unpublished schema and silently sends the original result on a mismatch. A new integration test drives the CLI bundled with claude-agent-sdk against a scripted, local Messages API and asserts what the model receives, and a new CI job runs it with the extra installed.

🤖 Generated with Claude Code

…result_rules=)

After the handler returns, a second Guard call screens its result before
the model reads it. A DENY, or an unevaluated policy under the default
on_guard_error="deny", returns the existing denial envelope instead of the
result. The check is labelled result_action, which defaults to
{action}.result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@arcjet-review arcjet-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arcjet Review — 🔴 High Risk

Decision: Needs Review

Rationale: This PR changes security-sensitive guard behavior by adding a second policy evaluation path over tool results, including fail-open/fail-closed handling, policy labels, metadata, and prompt-injection screening inputs. The implementation is well covered by unit and integration tests and I did not identify hardcoded secrets or an obvious injection vulnerability in the code paths shown. However, because the change directly affects security enforcement semantics and includes adversarial prompt-injection text in PR content, it should receive human security review before merging.

Summary of Changes

Adds result_rules and result_action to guard_tool so authored Claude Agent SDK tool results can be screened after the handler runs and before the model receives the result. The PR adds result text extraction, a second Guard evaluation path, fail-closed handling by default, README documentation, and tests for allow, deny, unavailable, invalid configuration, and label override behavior.

Security Flags

  • [LOW] Prompt injection in PR content (tests/integration/guard/test_claude_agent_sdk.py:107): The diff contains the phrase "ignore previous instructions" as a test fixture. It appears to be intentional prompt-injection test data, but it was ignored as an instruction and should be verified by a human reviewer under the adversarial PR-content policy.

Review Focus Areas

Review: 38eb094c | Model: openai/gpt-5.5 | Powered by Arcjet Review

@arcjet-review arcjet-review Bot added needs review Awaiting human review and removed ai-review-in-progress labels Oct 14, 2026
…s closed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@arcjet-rei

Copy link
Copy Markdown
Contributor Author

Replies to the Arcjet Review focus areas:

  • Untruncated text. Intended: screening a prefix lets an injection placed after the cut through. The text is sent whole; a request over the protocol limit, or a detector that returns no verdict, leaves the policy unevaluated, which denies under the default on_guard_error="deny". So an oversized result fails closed rather than being screened in part.
  • Resolver failure. Intended, and the same as rules on the call itself: Guard is still called without local rules so remote policy and the capture record it, and on_guard_error decides. Under the default that denies the result (test_rules_callable_raising_still_calls_guard_and_fails_closed).
  • Denial envelope in place of the result. It is the envelope a pre-call denial already returns for the same tool, so a consumer of that tool already handles it; only callers who opt into result_rules get it after the handler runs.
  • Non-sequence return. Added in 41a6f49: test_rules_callable_returning_no_bound_rules_fails_closed, over the shared NOT_BOUND_RULES cases (None, a string, a non-rule object, an unbound rule, a coroutine).
  • "ignore previous instructions" in the integration test. Deliberate test data: it is the tool's output the result screen is meant to catch.

arcjet-rei and others added 9 commits October 13, 2026 19:13
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aced with

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…one from the model

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n PostToolUse

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t a scripted Messages API

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@arcjet-rei arcjet-rei changed the title feat(guard): screen a Claude Agent SDK tool's result with guard_tool(result_rules=) feat(guard): screen Claude Agent SDK tool results before the model reads them Oct 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready Ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants