[Amaury Levé (Evangelink)]

Pull request

Summary

The test-quality workflow printed escaped HTML instead of a scorecard in this review on #12204. Update the shared prompt for automatic reviews and /review-tests to emit Markdown pipe tables and inline backticks, with proper literal-pipe escaping and all five assessment fields preserved.

Design and decision record

Maintained rationale and reproduction commands. The pinned gh-aw parser supports Markdown tables but treats raw HTML table markup as literal text. Use its supported format rather than modifying the renderer or weakening sanitization. This is a bounded bug fix, not a new feature requiring a separate design document.

Both reviewer locks were regenerated. Only their body-hash metadata changes; executable YAML, permissions, credentials, threat detection, and action pins are unchanged.

Demonstration

Before: the published review contains literal <table> markup and no rendered table. After: the real pinned-runtime fixture payload renders as a five-column scorecard with three correctly associated test/result rows and one additional row inside <details>.

The actual emitted payload was rendered through the GitHub Markdown API, then previewed locally in the original review DOM using GitHub's styles. Checked a 1280 x 721 viewport with 800- and 480-pixel review widths, long generic names, literal pipes, quotes and markup, and collapsed/expanded overflow. Cell content and associations were preserved without overlapping cells; narrow columns wrap heavily at 480 pixels. Empty grading scopes still omit the table.

This text record describes the local browser inspection; captures and raw logs remain local and have not been uploaded. The preview was explicitly labeled as unpublished. No existing review was edited, and no live model or safe-output publication run was dispatched.

Validation and regression proof

The committed patch at 849bfbc6428edc9728693c16c39e40a6057748cc matches the locally validated patch byte-for-byte. The comparison uses workflow sources archived from 6876cf113dce6e1db55092779e73840697a0af73, retaining the same new tests in both runs.

Toolchain: gh-aw v1.0.1, setup scripts at 221ba4f4319de6b8e809d1a15528b3ae6a818aa5, Python 3.12.10, Node.js v25.0.0, and ShellCheck 0.11.0. $actions is the unmodified pinned setup/js directory; $baseline is the isolated archive's .github\workflows directory; $cases is the two-method selection defined in the linked rationale.

Exact command	Result
python .github\scripts\test_gh_aw_runtime_contracts.py --actions-dir $actions --workflow-root $baseline @cases -v	Expected exit 1: two methods execute, the scorecard method fails for both consumers with escaped HTML, the escaped-cell method passes, no skips.
python .github\scripts\test_gh_aw_runtime_contracts.py --actions-dir $actions @cases -v	Exit 0: both methods pass, no skips.
python .github\scripts\test_gh_aw_runtime_contracts.py --actions-dir $actions -v	Exit 0: all 15 methods pass, no skips.
python .github\scripts\test_gh_aw_review_footer.py --scripts-directory $actions	Exit 0: two tests pass.
python .github\scripts\test_gh_aw_firewall_footer.py --module-path (Join-Path $actions 'firewall_blocked_domains.cjs')	Exit 0: seven tests pass.
gh aw compile test-reviewer-on-pr.agent test-reviewer.agent --strict --no-check-update --shellcheck	Exit 0: both workflows compile, 120 shell resources checked, zero warnings.
python .github\scripts\check_action_pins.py	Exit 0: 2,763 references across 59 files pass.
git diff --cached --check	Exit 0 before committing.

A parsed-YAML comparison and complete generated-file comparison excluding the first metadata line confirmed that both locks preserve every executable field. No .NET sources or packages changed. Hosted model behavior and review-publication permissions were not exercised.

Test-value evidence

Contract / test case	Without the prompt fix	Corrected prompt
GhAwRuntimeContractTests.test_authored_test_review_scorecard_publishes_as_markdown_table	Executed: detects the defect for both reviewer consumers because the real final API request contains <table> instead of a Markdown table.	Executed: passes for both consumers, asserting the complete scorecard, test/result associations, reviewed commit, advisory event, and attribution.
GhAwRuntimeContractTests.test_review_table_preserves_escaped_cells_and_collapsed_rows	Executed: passes; Markdown support already exists in the pinned renderer.	Executed: passes; long generic names, escaped pipes, markup, quotes, and collapsed overflow survive the publisher unchanged.

These are offline component executions with isolated GitHub API doubles, not live model or hosted review-publication proof. The linked rationale preserves the commands, observed failure, corrected results, and rendering limitations.

[Copilot]

🟢 Approval recommended

No active review issues remain for the change to renderer-compatible Markdown scorecards.

No open issues 🎉

What changed in this PR

Fixes scorecard rendering in automatic test-quality reviews and /review-tests by replacing raw HTML tables with Markdown supported by the pinned safe-output renderer.

Changes:

• Updates the shared prompt to use five-column Markdown tables, inline backticks, and escaped literal pipes.
• Adds runtime regression tests for scorecard publication, escaped cell content, and collapsed overflow.
• Documents the reproduction and validation evidence and refreshes both workflow body hashes without changing executable YAML.

Test-value evidence: For revision 849bfbc6428edc9728693c16c39e40a6057748cc, the PR reports that the scorecard regression fails against the baseline for both consumers and passes with the fix; all 15 runtime tests pass. These results were not independently executed here. Live model behavior and hosted review publication remain untested.

File	Description
.github/​workflows/​test-reviewer.agent.lock.yml	Refreshes the /review-tests workflow body hash.
.github/​workflows/​test-reviewer-on-pr.agent.lock.yml	Refreshes the automatic reviewer workflow body hash.
.github/​workflows/​shared/​test-reviewer-shared.md	Replaces HTML scorecards with Markdown tables and defines escaping rules.
.github/​workflows/​README.md	Records renderer constraints, reproduction commands, and validation limitations.
.github/​scripts/​test_gh_aw_runtime_contracts.py	Adds final review-payload assertions for scorecards and escaped overflow tables.

_{Review effort: Balanced}