diff --git a/.agents/skills/execution-playbooks/references/plan-execute-v1.md b/.agents/skills/execution-playbooks/references/plan-execute-v1.md new file mode 100644 index 00000000..9dbdf7f4 --- /dev/null +++ b/.agents/skills/execution-playbooks/references/plan-execute-v1.md @@ -0,0 +1,34 @@ +# Execution playbook: `plan-execute@1` + +## Accepted brief form + +This contract accepts `strike` only, and it applies when the resolved worker lane is an execution-class model: a lower-context lane that carries out a plan instead of designing one. +The brief must carry a complete `## Execution plan` section, materialized from the planning artifact - the TLC `data//tasks/T0X.md` contract when one exists - and `bin/sq-plan-validate.sh` structurally validates it before dispatch. +The planner owns the design; the executing worker receives the plan, not the sources it was derived from. + +## Required sequence + +1. Accept a brief whose `## Execution plan` section is complete. The section must carry these five labels, each written as a label line followed by its list entries: `Files to touch` (at least one path-like entry, i.e. real file paths), `Ordered steps` (steps numbered, e.g. `1.`), `Acceptance criteria`, `Verification commands` (a command to run), and `Out of scope` (an explicit out-of-scope statement). +2. Execute the plan as written; do not redesign, re-scope, or re-plan the change. +3. Follow `tlc-implement` for the implementation method and write the checklist under `data//artifacts/`. +4. Stop with `blocked:` when the plan is incomplete, internally contradictory, or contradicted by the code, naming the exact gap rather than filling it by inference. +5. Report acceptance evidence and the verification command result; leave review, fixes, delivery, and merge to the existing owners. + +## Required evidence + +- The brief's materialized `## Execution plan` section, with every required field present and non-empty. +- A checklist under `data//artifacts/` recording the plan fields as realized: files touched as planned, steps executed, acceptance met, and the verification command run. +- A `blocked:` report naming the missing or contradictory plan element when execution cannot proceed safely. + +## Exit predicate + +Every planned file and step is realized or explicitly reported, acceptance criteria are met, and the planned verification command has been run with its result recorded. + +## Stop conditions + +Stop when the plan is incomplete or internally contradictory, when a step contradicts the current code, when an unplanned design decision would be required, or when the change would materially exceed the materialized plan. + +## Ownership and anti-patterns + +The planner (with the commander) owns design, and `AGENTS.md` section 7 plus `execution-playbooks` own selection; this contract only carries the materialized plan into execution. +Do not redesign the change, silently widen scope, edit or defer the plan instead of executing it, or create playbook-owned review, delivery, approval, or terminal authority. diff --git a/AGENTS.md b/AGENTS.md index b3ba51dd..bcca8cd6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -358,6 +358,7 @@ Once ownership is settled, validate exactly once against that final head so no o An ask-user finding returns as `needs-decision`; Squad decides only when the configured authority permits, otherwise escalates to the commander. For a strike brief with an explicit execution playbook, run its structural validator before sending the implementation to drill or the selected delivery path; the playbook validator never replaces drill. +When the resolved worker lane is an execution-class model that must not design, select `plan-execute@1` with the plan materialized from the planning artifact and run `bin/sq-plan-validate.sh ` before dispatch; `bin/sq-spawn.sh` refuses an incomplete plan. Versioned planning and evaluation playbooks are methods only, while delivery remains owned by mode and drill and merge remains commander-authorized. `orchestrate` is rejected because Commander, XO, backlog, and supervision already own programme coordination; `autopilot-full`, `autopilot-stack`, and `autonomous-run` remain deferred until their documented measurable triggers are proven, and none adds merge, discard, or destructive authority. `arena` means competing approaches to one problem with one selected base, `swarm` means parallel coverage of distinct slices, and `interrogate` means independent attacks on one artifact with deduplication and judgment; these are auxiliary topologies, not playbook identities, dispatch owners, or inferences from free-form `sq-tasks` `kind`. diff --git a/bin/sq-brief.sh b/bin/sq-brief.sh index 7c0b65fd..8a6a740b 100755 --- a/bin/sq-brief.sh +++ b/bin/sq-brief.sh @@ -187,6 +187,7 @@ if [ "$PLAYBOOK_SET" -eq 1 ]; then visual-parity@1) PLAYBOOK_ID=visual-parity; PLAYBOOK_VERSION=1; PLAYBOOK_SECTION=$(cat "$SQUAD_ROOT/.agents/skills/execution-playbooks/references/visual-parity-v1.md"); EXPECTED_KIND='recon|strike' ;; multi-phase-plan@1) PLAYBOOK_ID=multi-phase-plan; PLAYBOOK_VERSION=1; PLAYBOOK_SECTION=$(cat "$SQUAD_ROOT/.agents/skills/execution-playbooks/references/multi-phase-plan-v1.md"); EXPECTED_KIND=recon ;; "eval@1") PLAYBOOK_ID='eval'; PLAYBOOK_VERSION=1; PLAYBOOK_SECTION=$(cat "$SQUAD_ROOT/.agents/skills/execution-playbooks/references/eval-v1.md"); EXPECTED_KIND=recon ;; + plan-execute@1) PLAYBOOK_ID=plan-execute; PLAYBOOK_VERSION=1; PLAYBOOK_SECTION=$(cat "$SQUAD_ROOT/.agents/skills/execution-playbooks/references/plan-execute-v1.md"); EXPECTED_KIND=strike ;; orchestrate@1) echo "error: orchestrate@1 disposition=reject; Commander, XO, backlog, and supervision already own programme coordination" >&2; exit 1 ;; autopilot-full@1|autopilot-stack@1) echo "error: $PLAYBOOK disposition=defer; re-evaluate only for a real multi-PR programme where commander merge approval is the measured bottleneck and yolo cannot cover it; no auto-merge or drill bypass" >&2; exit 1 ;; autonomous-run@1) echo "error: autonomous-run disposition=defer; re-evaluate only with a real task whose cycle the current supervision cannot conduct, plus a verifiable terminal predicate, budget, stop conditions, and duplicate prevention" >&2; exit 1 ;; diff --git a/bin/sq-plan-validate.sh b/bin/sq-plan-validate.sh new file mode 100755 index 00000000..2630aa95 --- /dev/null +++ b/bin/sq-plan-validate.sh @@ -0,0 +1,106 @@ +#!/usr/bin/env bash +# Deterministic pre-dispatch structural gate for a brief's execution plan. +# Usage: sq-plan-validate.sh +# +# Reads data//brief.md, bounds its `## Execution plan` section, and refuses +# when a required field is absent or structurally empty. Required fields and +# shapes (label line, then its entries as a list; short fields may carry inline +# content after the label): +# - Files to touch, with at least one path-like entry +# - Ordered steps, with at least one numbered step +# - Acceptance criteria +# - Verification command(s) +# - Out of scope +# The check is structural only: it never judges plan quality, correctness, or +# feasibility. It is the dispatch-time companion to bin/sq-playbook-validate.sh, +# which owns the post-work checklist evidence for the plan-execute@1 playbook. +set -eu +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +BASE="${SQUAD_BASE:-${SQUAD_HOME:-$ROOT}}" +DATA="${SQUAD_DATA_OVERRIDE:-$BASE/data}" +ID=${1:-} +[ -n "$ID" ] || { echo "error: task id is required" >&2; exit 2; } +BRIEF="$DATA/$ID/brief.md" +[ -f "$BRIEF" ] || { echo "missing: $BRIEF"; exit 1; } + +awk -v id="$ID" ' +function label_text(l) { + sub(/^[[:space:]]+/, "", l) + sub(/^[*_]+/, "", l) + sub(/[*_]+[[:space:]]*$/, "", l) + return l +} +function field(l) { + if (l ~ /^[Ff]iles[[:space:]]+to[[:space:]]+touch/) return "files" + if (l ~ /^([Oo]rdered[[:space:]]+)?[Ss]teps/) return "steps" + if (l ~ /^[Aa]cceptance[[:space:]]+criteria/) return "acceptance" + if (l ~ /^[Vv]erification[[:space:]]+commands?/) return "verification" + if (l ~ /^[Oo]ut[[:space:]]+of[[:space:]]+scope/) return "out-of-scope" + return "" +} +function flush() { + if (cur != "") { seen[cur] = 1; body[cur] = body[cur] "\n" text } +} +BEGIN { inplan = 0; heading = 0; cur = ""; text = "" } +/^##[[:space:]]+[Ee]xecution[[:space:]]+[Pp]lan[[:space:]]*$/ { inplan = 1; heading = 1; next } +inplan && /^#[[:space:]]/ { inplan = 0; next } +inplan && /^##[[:space:]]/ { inplan = 0; next } +!inplan { next } +{ + l = label_text($0) + f = field(l) + if (f != "") { + flush() + cur = f; text = ""; inline[cur] = "" + if (l ~ /:/) { + sub(/^[^:]*:/, "", l) + gsub(/^[[:space:]]+|[[:space:]]+$/, "", l) + inline[cur] = l + } + next + } + if (cur != "") { + text = text "\n" $0 + item = $0 + sub(/^[[:space:]]+/, "", item) + if (item ~ /^[-*+][[:space:]]/ || item ~ /^[0-9]+[.)]/) has_item[cur] = 1 + if (item ~ /^[0-9]+[.)]/) has_step[cur] = 1 + } +} +END { + flush() + if (!heading) { print "missing: execution plan section (## Execution plan)"; exit 1 } + labels["files"] = "files to touch (exact paths)" + labels["steps"] = "ordered steps" + labels["acceptance"] = "acceptance criteria" + labels["verification"] = "verification command" + labels["out-of-scope"] = "out of scope" + split("files steps acceptance verification out-of-scope", keys, " ") + bad = 0 + for (i = 1; i <= 5; i++) { + k = keys[i] + if (!(k in seen)) { + printf "missing: execution plan field \"%s\"\n", labels[k] + bad = 1 + continue + } + combined = inline[k] "\n" body[k] + if (inline[k] == "" && has_item[k] != 1) { + printf "empty: execution plan field \"%s\"\n", labels[k] + bad = 1 + continue + } + if (k == "files" && combined !~ /\/|[A-Za-z0-9_-]+\.[A-Za-z0-9]+/) { + printf "invalid: execution plan field \"%s\" has no path-like entry\n", labels[k] + bad = 1 + } + if (k == "steps" && has_step[k] != 1 && inline[k] !~ /^[0-9]+[.)]/) { + printf "invalid: execution plan field \"%s\" has no numbered step\n", labels[k] + bad = 1 + } + } + if (bad) exit 1 + printf "execution plan structurally valid: %s\n", id +} +' "$BRIEF" diff --git a/bin/sq-playbook-validate.sh b/bin/sq-playbook-validate.sh index 3b851313..edf0a520 100755 --- a/bin/sq-playbook-validate.sh +++ b/bin/sq-playbook-validate.sh @@ -15,7 +15,7 @@ ARTIFACTS="$DATA/$ID/artifacts" PLAYBOOK=$(sed -n 's/^playbook=//p' "$META" | head -n 1) VERSION=$(sed -n 's/^playbook_version=//p' "$META" | head -n 1) case "$PLAYBOOK@$VERSION" in - bug-fix@1|investigation@1|feature@1|refactoring@1|prototype@1|perf@1|hillclimb@1|runtime-forensics@1|trace-forensics@1|visual-parity@1|multi-phase-plan@1|eval@1) ;; + bug-fix@1|investigation@1|feature@1|refactoring@1|prototype@1|perf@1|hillclimb@1|runtime-forensics@1|trace-forensics@1|visual-parity@1|multi-phase-plan@1|eval@1|plan-execute@1) ;; *) echo "error: task $ID has unsupported execution playbook identity"; exit 1 ;; esac CHECKLIST= @@ -172,6 +172,16 @@ case "$PLAYBOOK@$VERSION" in "recommendation;promote|reject;production|promotion" ) ;; + plan-execute@1) + CRITERIA_LABELS=("files touched as planned" "steps executed" "acceptance met" "verification run") + CRITERIA_PATTERNS=("files" "steps" "acceptance" "verification") + CRITERIA_CHECKS=( + "files|paths;planned|materialized" + "steps;executed|followed" + "acceptance;met|verified|criteria" + "verification;run|executed|result" + ) + ;; esac COUNT=${#CRITERIA_LABELS[@]} # Collect all proofs first for duplicate detection. diff --git a/bin/sq-spawn.sh b/bin/sq-spawn.sh index 8e1399db..e17afeb5 100755 --- a/bin/sq-spawn.sh +++ b/bin/sq-spawn.sh @@ -1452,11 +1452,21 @@ if [ "$PLAYBOOK_LINES" -eq 1 ]; then 'Execution playbook: id=feature version=1') PLAYBOOK_META=feature; PLAYBOOK_VERSION_META=1; EXPECTED_KIND=strike ;; 'Execution playbook: id=refactoring version=1') PLAYBOOK_META=refactoring; PLAYBOOK_VERSION_META=1; EXPECTED_KIND=strike ;; 'Execution playbook: id=prototype version=1') PLAYBOOK_META=prototype; PLAYBOOK_VERSION_META=1; EXPECTED_KIND=recon ;; + 'Execution playbook: id=plan-execute version=1') PLAYBOOK_META=plan-execute; PLAYBOOK_VERSION_META=1; EXPECTED_KIND=strike ;; *) echo "error: malformed or unsupported execution playbook identity in $BRIEF" >&2; exit 1 ;; esac [ "$KIND" = "$EXPECTED_KIND" ] || { echo "error: $PLAYBOOK_META@1 is compatible only with kind: $EXPECTED_KIND" >&2; exit 1; } # shellcheck disable=SC2016 # Backticks are literal brief syntax. grep -q "^# Execution playbook: \`$PLAYBOOK_META@$PLAYBOOK_VERSION_META\`$" "$BRIEF" || { echo "error: $PLAYBOOK_META@$PLAYBOOK_VERSION_META identity has no materialized contract" >&2; exit 1; } + # A plan-execute brief carries the design as a materialized plan: the structural + # gate refuses dispatch before any endpoint exists when that plan is incomplete. + if [ "$PLAYBOOK_META" = plan-execute ]; then + if ! plan_validate_out=$(SQUAD_BASE="$SQUAD_BASE" "$SCRIPT_DIR/sq-plan-validate.sh" "$ID" 2>&1); then + printf '%s\n' "$plan_validate_out" >&2 + echo "error: $ID brief execution plan failed structural validation; materialize a complete plan before dispatch" >&2 + exit 1 + fi + fi fi delivery_rigor_rank() { # -> 3 (most rigor) .. 1 (least); 0 = not a task mode diff --git a/docs/documentation-audiences.json b/docs/documentation-audiences.json index f5b97f29..b712937e 100644 --- a/docs/documentation-audiences.json +++ b/docs/documentation-audiences.json @@ -1564,6 +1564,10 @@ "path": ".agents/skills/execution-playbooks/references/eval-v1.md", "audience": "agent-runtime" }, + { + "path": ".agents/skills/execution-playbooks/references/plan-execute-v1.md", + "audience": "agent-runtime" + }, { "path": "skills/interview-me/SKILL.md", "audience": "public-product" diff --git a/docs/pt-BR/scripts.md b/docs/pt-BR/scripts.md index 417d260e..70320d6b 100644 --- a/docs/pt-BR/scripts.md +++ b/docs/pt-BR/scripts.md @@ -127,7 +127,7 @@ A recusa compartilhada do gate drill para entrypoints do ciclo de vida da unidad | `sq-pr-check-migrate.sh` | Quarentenair polls de tarefa antigos sem execução e reconstruir apenas polls canônicos | | `sq-pr-check.sh` | Registrar valores validados de `pr=` e `pr_head=`, então armar atomicamente um poll estático de merge | | `sq-pr-merge.sh` | Registrar metadados do PR, então mesclar a URL completa canônica GitHub do PR da tarefa | -| `sq-playbook-validate.sh` | Validar evidência estrutural de um playbook de execução materializado (bug-fix@1, investigation@1, feature@1, refactoring@1, prototype@1, perf@1, hillclimb@1, runtime-forensics@1, trace-forensics@1, visual-parity@1, multi-phase-plan@1, eval@1) | +| `sq-playbook-validate.sh` | Validar evidência estrutural de um playbook de execução materializado (bug-fix@1, investigation@1, feature@1, refactoring@1, prototype@1, perf@1, hillclimb@1, runtime-forensics@1, trace-forensics@1, visual-parity@1, multi-phase-plan@1, eval@1, plan-execute@1) | | `sq-promote.sh` | Promover uma tarefa recon in-place a uma tarefa strike protegida com modo de entrega explícito | | `sq-teardown.sh` | Teardown fail-closed: devolver worktrees ship landadas, exigir entregáveis completos de recon, aposentar bases XO | | `sq-harness.sh` | Detectar o harness em execução e resolver crew ou XO harness, modelo e esforço | diff --git a/docs/verification/playbook-absorptions.md b/docs/verification/playbook-absorptions.md index e1c67e5c..07cd0a29 100644 --- a/docs/verification/playbook-absorptions.md +++ b/docs/verification/playbook-absorptions.md @@ -20,7 +20,7 @@ On 2026-09-19, the focused `sq-brief` test exercised `--playbook @1` for e sq-brief.sh: absorbed playbook names are refused and leave no selectable brief ``` -The test is [`test_absorbed_playbook_names_are_not_selectable`](../../tests/sq-brief.test.sh), and the registry contains wave-1 contracts `bug-fix@1`, `feature@1`, `investigation@1`, `prototype@1`, `refactoring@1` plus wave-2 contracts `perf@1`, `hillclimb@1`, `runtime-forensics@1`, `trace-forensics@1`, and `visual-parity@1` plus planning/evaluation methods `multi-phase-plan@1` and `eval@1` under `.agents/skills/execution-playbooks/references/`. +The test is [`test_absorbed_playbook_names_are_not_selectable`](../../tests/sq-brief.test.sh), and the registry contains wave-1 contracts `bug-fix@1`, `feature@1`, `investigation@1`, `prototype@1`, `refactoring@1` plus wave-2 contracts `perf@1`, `hillclimb@1`, `runtime-forensics@1`, `trace-forensics@1`, and `visual-parity@1` plus planning/evaluation methods `multi-phase-plan@1` and `eval@1` plus the materialized-plan execution contract `plan-execute@1` under `.agents/skills/execution-playbooks/references/`. The focused test command is: @@ -34,7 +34,7 @@ The registry inventory command is: find .agents/skills/execution-playbooks -maxdepth 2 -type f -print | sort ``` -Its expected output contains `SKILL.md` and the reference files for all twelve selectable execution playbook contracts. +Its expected output contains `SKILL.md` and the reference files for all thirteen selectable execution playbook contracts. ## Deferred and auxiliary policy @@ -49,4 +49,4 @@ The three auxiliary topologies are not execution playbook identities or dispatch `arena` competes on one problem and selects one base, `swarm` covers distinct slices, and `interrogate` independently attacks one artifact with deduplication and judgment. For PR or diff surfaces, `interrogate` points to the read-only Drill surface documented in [`docs/pr-review.md`](../pr-review.md), and separate reviews are limited to requested or knowledge-only review deliverables. -The catalog remains 22 playbooks because these topology names and the four refused or deferred names are not selectable identities. +The catalog now counts 23 playbook identities: 13 selectable contracts, 5 absorbed upstream names, and 5 refused or deferred names. diff --git a/tests/sq-brief.test.sh b/tests/sq-brief.test.sh index 38c52220..c0c6c221 100755 --- a/tests/sq-brief.test.sh +++ b/tests/sq-brief.test.sh @@ -405,6 +405,19 @@ test_remaining_lifecycle_playbooks() { assert_grep "not a layer-by-layer" "$brief" "multi-phase-plan contract missing layer boundary" assert_grep "existing backlog" "$brief" "multi-phase-plan contract missing backlog handoff" + SQUAD_BASE="$home" "$ROOT/bin/sq-brief.sh" lifecycle-plan-execute repo --mode drill --playbook plan-execute@1 >/dev/null 2>&1 || fail "plan-execute@1 should materialize as strike" + brief="$home/data/lifecycle-plan-execute/brief.md" + assert_grep "Execution playbook: id=plan-execute version=1" "$brief" "plan-execute identity missing" + assert_grep "execution-class" "$brief" "plan-execute contract missing execution-class scope" + assert_grep "materialized from the planning artifact" "$brief" "plan-execute contract missing the plan materialization rule" + assert_grep "tlc-implement" "$brief" "plan-execute contract missing tlc-implement" + assert_grep "data//artifacts/" "$brief" "plan-execute contract missing the checklist location" + assert_grep "blocked:" "$brief" "plan-execute contract missing the stop-with-blocked rule" + out=$(SQUAD_BASE="$home" "$ROOT/bin/sq-brief.sh" lifecycle-plan-execute-recon repo --recon --playbook plan-execute@1 2>&1); status=$? + [ "$status" -ne 0 ] || fail "plan-execute@1 must refuse recon" + assert_contains "$out" "accepts only kind: strike" "plan-execute refusal should name strike compatibility" + assert_absent "$home/data/lifecycle-plan-execute-recon/brief.md" "plan-execute recon refusal must not leave a partial brief" + SQUAD_BASE="$home" "$ROOT/bin/sq-brief.sh" lifecycle-eval repo --recon --playbook eval@1 >/dev/null 2>&1 || fail "eval@1 should materialize" brief="$home/data/lifecycle-eval/brief.md" assert_grep "Execution playbook: id=eval version=1" "$brief" "eval identity missing" @@ -412,7 +425,7 @@ test_remaining_lifecycle_playbooks() { assert_grep "chain-elicitation" "$brief" "eval contract missing chain-elicitation prevention" assert_grep "does not enter production" "$brief" "eval contract missing promotion boundary" - for playbook in shipping@2 multi-phase-plan@2 eval@2; do + for playbook in shipping@2 multi-phase-plan@2 eval@2 plan-execute@2; do out=$(SQUAD_BASE="$home" "$ROOT/bin/sq-brief.sh" "lifecycle-invalid-$RANDOM" repo --mode drill --playbook "$playbook" 2>&1); status=$? [ "$status" -ne 0 ] || fail "$playbook should be refused" assert_contains "$out" "unknown execution playbook" "$playbook should identify invalid identity" diff --git a/tests/sq-plan-validate.test.sh b/tests/sq-plan-validate.test.sh new file mode 100755 index 00000000..a720fd4c --- /dev/null +++ b/tests/sq-plan-validate.test.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# shellcheck disable=SC2016 +# SC2016 off for this file: Markdown backticks are literal test fixtures. +# Focused behavior tests for the plan-execute@1 pre-dispatch structural gate. +# +# Covers bin/sq-plan-validate.sh field detection and the sq-spawn.sh refusal it +# backs, exercising both through their command-line interfaces. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +TMP_ROOT=$(fm_test_tmproot sq-plan-validate) +HOME="$TMP_ROOT/home" +ID=plan-execute-test +mkdir -p "$HOME/data/$ID" "$HOME/state" "$HOME/projects/alpha" +BRIEF="$HOME/data/$ID/brief.md" +VALIDATOR="$ROOT/bin/sq-plan-validate.sh" + +# plan_block - emit one required field's label and body lines. +plan_block() { + case "$1" in + files) printf '%s\n' 'Files to touch (exact paths):' '- `bin/sq-plan-validate.sh` (new)' '- `AGENTS.md`' ;; + steps) printf '%s\n' 'Ordered steps:' '1. Read the sources.' '2. Implement the validator.' ;; + acceptance) printf '%s\n' 'Acceptance criteria:' '- A complete plan passes.' ;; + verification) printf '%s\n' 'Verification commands:' '- `bin/sq-lint.sh`' ;; + out-of-scope) printf '%s\n' 'Out of scope:' '- No change to existing playbook criteria.' ;; + esac +} + +# write_plan [omit-field] [blank-field] +# omit-field: drop that field entirely. blank-field: keep its label, no body. +write_plan() { + local file=$1 omit=${2:-} blank=${3:-} f + { + printf '%s\n' '# Task' '' '## Execution plan' '' + for f in files steps acceptance verification out-of-scope; do + [ "$f" = "$omit" ] && continue + if [ "$f" = "$blank" ]; then + plan_block "$f" | head -n 1 + continue + fi + plan_block "$f" + done + printf '%s\n' '' 'Constraints: repository requires the drill signature.' '' '# Next section' 'not part of the plan' + } > "$file" +} + +# --------------------------------------------------------------------------- +# A complete plan passes. +# --------------------------------------------------------------------------- +write_plan "$BRIEF" +out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? +expect_code 0 "$rc" "a complete execution plan should validate" +assert_contains "$out" "execution plan structurally valid" "validator should report a successful validation" + +# --------------------------------------------------------------------------- +# Each field omitted entirely fails with its own named message, and the other +# fields are not reported as missing. +# --------------------------------------------------------------------------- +while IFS='|' read -r field label; do + write_plan "$BRIEF" "$field" + out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? + [ "$rc" -ne 0 ] || fail "an omitted '$field' field should fail" + assert_contains "$out" "missing: execution plan field \"$label\"" "omitted '$field' was not named" + while IFS='|' read -r other otherlabel; do + [ "$other" = "$field" ] && continue + assert_not_contains "$out" "execution plan field \"$otherlabel\"" "omitted '$field' wrongly reported '$other'" + done <<'FIELDS' +files|files to touch (exact paths) +steps|ordered steps +acceptance|acceptance criteria +verification|verification command +out-of-scope|out of scope +FIELDS +done <<'FIELDS' +files|files to touch (exact paths) +steps|ordered steps +acceptance|acceptance criteria +verification|verification command +out-of-scope|out of scope +FIELDS + +# --------------------------------------------------------------------------- +# A field present but structurally empty fails as empty, not missing. +# --------------------------------------------------------------------------- +while IFS='|' read -r field label; do + write_plan "$BRIEF" '' "$field" + out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? + [ "$rc" -ne 0 ] || fail "an empty '$field' field should fail" + assert_contains "$out" "empty: execution plan field \"$label\"" "empty '$field' was not named as empty" +done <<'FIELDS' +files|files to touch (exact paths) +steps|ordered steps +acceptance|acceptance criteria +verification|verification command +out-of-scope|out of scope +FIELDS + +# --------------------------------------------------------------------------- +# Structural depth: files need a path-like entry and steps need a numbered step. +# --------------------------------------------------------------------------- +{ + printf '%s\n' '# Task' '' '## Execution plan' + printf '%s\n' 'Files to touch (exact paths):' '- somewhere in the codebase' + plan_block steps + plan_block acceptance + plan_block verification + plan_block out-of-scope +} > "$BRIEF" +out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? +[ "$rc" -ne 0 ] || fail "a files field without a path-like entry should fail" +assert_contains "$out" "no path-like entry" "missing path-like entry was not named" +{ + printf '%s\n' '# Task' '' '## Execution plan' + plan_block files + printf '%s\n' 'Ordered steps:' '- just do it' + plan_block acceptance + plan_block verification + plan_block out-of-scope +} > "$BRIEF" +out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? +[ "$rc" -ne 0 ] || fail "a steps field without a numbered step should fail" +assert_contains "$out" "no numbered step" "missing numbered step was not named" + +# --------------------------------------------------------------------------- +# A brief with no execution plan section is refused by name. +# --------------------------------------------------------------------------- +printf '%s\n' '# Task' '' 'no plan here' > "$BRIEF" +out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? +expect_code 1 "$rc" "a brief without an execution plan should fail" +assert_contains "$out" "missing: execution plan section" "missing plan section was not named" + +# A plan section ends at the next heading: a required field supplied only under +# a later heading does not satisfy the plan. +{ + printf '%s\n' '# Task' '' '## Execution plan' + plan_block files + plan_block steps + plan_block acceptance + plan_block out-of-scope + printf '%s\n' '' '## Notes' '' 'Verification commands:' '- `bin/sq-lint.sh`' +} > "$BRIEF" +out=$(SQUAD_BASE="$HOME" "$VALIDATOR" "$ID" 2>&1); rc=$? +[ "$rc" -ne 0 ] || fail "a required field under a later heading must not satisfy the plan" +assert_contains "$out" 'missing: execution plan field "verification command"' "later-heading field wrongly satisfied the plan" + +# --------------------------------------------------------------------------- +# sq-spawn.sh refuses a plan-execute@1 brief whose plan is incomplete before any +# endpoint or metadata exists. Only the playbook gate is under test here; the +# refusal lands before backend selection. An explicit --harness pins resolution +# so the refusal never depends on ambient harness detection (CI has none). +# --------------------------------------------------------------------------- +SPAWN_ID=plan-execute-spawn +mkdir -p "$HOME/data/$SPAWN_ID" "$HOME/projects/alpha" +cat > "$HOME/data/$SPAWN_ID/brief.md" <<'EOF' +Execution playbook: id=plan-execute version=1 +# Execution playbook: `plan-execute@1` +Delivery contract: mode=drill +The operator must report status: echo '{state}: {note}' >> 'state/task.status' +## Execution plan +Files to touch (exact paths): +- `bin/sq-plan-validate.sh` +Ordered steps: +1. Implement the validator. +EOF +out=$(SQUAD_ROOT_OVERRIDE='' SQUAD_BASE="$HOME" SQUAD_STATE_OVERRIDE="$HOME/state" \ + SQUAD_DATA_OVERRIDE="$HOME/data" SQUAD_PROJECTS_OVERRIDE="$HOME/projects" \ + SQUAD_CONFIG_OVERRIDE="$HOME/config" SQUAD_BACKEND=tmux SQUAD_SPAWN_NO_GUARD=1 TMUX='' \ + "$ROOT/bin/sq-spawn.sh" "$SPAWN_ID" projects/alpha --mode drill --yolo off --harness pi 2>&1); rc=$? +[ "$rc" -ne 0 ] || fail "spawn should refuse a plan-execute@1 brief with an incomplete plan" +assert_contains "$out" 'missing: execution plan field "acceptance criteria"' "spawn refusal omitted the validator's named failure" +assert_contains "$out" "failed structural validation" "spawn refusal did not name the plan gate" +assert_absent "$HOME/state/$SPAWN_ID.meta" "a refused plan-execute spawn must not create task metadata" + +pass "sq-plan validator: complete plans pass, every missing field is named, and spawn refuses an incomplete plan" diff --git a/tests/sq-playbook-validator.test.sh b/tests/sq-playbook-validator.test.sh index 913723e1..56592179 100755 --- a/tests/sq-playbook-validator.test.sh +++ b/tests/sq-playbook-validator.test.sh @@ -165,7 +165,7 @@ for playbook in perf hillclimb runtime-forensics trace-forensics visual-parity; [ "$rc" -ne 0 ] || fail "$playbook non-comparable or incomplete evidence should fail" assert_contains "$out" "quality" "$playbook refusal should identify an evidence quality failure" done -for playbook in multi-phase-plan eval; do +for playbook in multi-phase-plan eval plan-execute; do id="lifecycle-$playbook" mkdir -p "$HOME/data/$id/artifacts" printf 'playbook=%s\nplaybook_version=1\n' "$playbook" > "$HOME/state/$id.meta" @@ -176,6 +176,8 @@ for playbook in multi-phase-plan eval; do printf '%s\n' '## Criterion 1' 'Proof: outcome and real dependencies command path' '## Criterion 2' 'Proof: unit verification command path' '## Criterion 3' 'Proof: not a layer task decomposition command path' '## Criterion 4' 'Proof: existing backlog handoff queue command path' '## Criterion 5' 'Proof: next action unresolved question command path' ;; eval) printf '%s\n' '## Criterion 1' 'Proof: baseline variant metric rubric command path' '## Criterion 2' 'Proof: sanitized candidate-visible path directory cue command path' '## Criterion 3' 'Proof: blinded cases judge command path' '## Criterion 4' 'Proof: chain elicitation prevention stop command path' '## Criterion 5' 'Proof: recommendation promote or reject production promotion command path' ;; + plan-execute) + printf '%s\n' '## Criterion 1' 'Proof: files touched as planned path: bin/x.sh command: git status' '## Criterion 2' 'Proof: steps executed in order command: run the plan' '## Criterion 3' 'Proof: acceptance criteria met command: verify behavior' '## Criterion 4' 'Proof: verification command run result path: out.log' ;; esac } > "$HOME/data/$id/artifacts/checks.md" out=$(SQUAD_BASE="$HOME" "$ROOT/bin/sq-playbook-validate.sh" "$id" 2>&1); rc=$?