Skip to content

feat(pipeline): order active stages by the WSJF judgement Facility already records - #176

Merged
adrian-lorenzo merged 6 commits into
theam:mainfrom
vicente-debug:feat/109-order-backlog-by-wsjf
Aug 28, 2026
Merged

feat(pipeline): order active stages by the WSJF judgement Facility already records#176
adrian-lorenzo merged 6 commits into
theam:mainfrom
vicente-debug:feat/109-order-backlog-by-wsjf

Conversation

@vicente-debug

Copy link
Copy Markdown
Contributor

Closes #109.

What this does

The board no longer sorts active stages by gh_updated_at. It sorts them by the WSJF judgement Facility already records — and then threw away. The issue's own framing ("Facility forms an opinion about priority, and then throws it away") is the design: this PR adds no new opinion, it consumes the existing one.

  • assemblePipelineStories reads the ## Value section back out of the mirrored issue body (gh_issues.body_md) and scores it with the existing wsjfScore — the first caller either exported function has had outside its unit test.
  • Active stages sort scored stories first, highest score on top. Unscored stories sit below, in GitHub's own newest-created-first order. Shipped keeps recency: it is a log of what left the pipeline, not a queue, and "most recently shipped on top" is the honest order there.
  • The score and its components ride PipelineStorySchema and StoryDetailSchema, and the board renders a wsjf 3.75 chip whose tooltip is the breakdown (value 8 · time 5 · risk 2 · effort 4) — the position is inspectable, not merely asserted.
  • The serialisation the PO-task executor used to build inline moves into @facility/harness (wsjfValueSection) next to its new inverse (parseWsjfValueSection), so the round trip through the GitHub mirror is pinned by one pair of functions and one pair of tests.

Where the score lives — the first open question, answered

In the issue body on GitHub, where executeTaskCreation already writes it. That choice does the most work per line of code:

  • No second source of truth. #101's reasoning about assignees applies verbatim: a Facility-native priority field would have no writer, no sync path and no reconciliation story. The body block has all three today — written on task acceptance, mirrored on every webhook and sync, echoed back within seconds of an edit.
  • The human override falls out for free, with a record. Editing the JSON block on GitHub is an attributed, timestamped issue edit — it lands in the issue history and in Facility's mirror without any new machinery. That is "an override is a decision with a reason, in the record", minus a bespoke approvals flow.
  • Hand-opened issues can opt in by adding the same ## Value block the PO agent writes. Without one they are simply unscored, which is handled explicitly, not accidentally: always below scored stories, in stable creation order, with the chip's absence marking the boundary.

Malformed blocks (bad JSON, missing components, effort <= 0, a fence that actually belongs to the next section) parse to unscored, never to an error — the majority of issues are hand-written, and a parser that can be angered by prose would be a liability.

What good looks like, checked against the issue

  • "Priority survives an issue being touched." Nothing in the active-stage comparator reads gh_updated_at any more. Triggering an agent — the acknowledgement comment plus the Show the signed-in GitHub user, and assign them on the issue and the PR #70 self-assignment that made the board reorder in response to your own actions — now moves nothing.
  • "The judgement is inspectable." The chip and its breakdown, plus the block itself in the story body.
  • "Unscored stories are handled explicitly." Deliberately below scored ones, newest-created first; tested.
  • "Re-scoring is visible when it happens." A re-score is an issue edit: it shows up in the issue history, the timeline's issue_updated, and the mirror — not as a silent overnight rearrangement.

Deliberately left open

The issue's larger questions — PO re-scoring cadence, decay, whether a re-prioritisation must pass through Approvals, effort informed by measured run cost, the #110 dependency graph — are policy, and the issue itself says the design belongs to whoever implements it. This PR keeps the mechanism small enough that any of those answers can be built on top of it: they all reduce to "who is allowed to edit the ## Value block, and when", because after this change the block is finally load-bearing.

Tests

  • packages/harness: round-trip through wsjfValueSection/parseWsjfValueSection, plus the malformed-body taxonomy.
  • services/api: a pipeline test where a stale scored story outranks a freshly-commented scored one, unscored stories keep creation order despite activity bumps, and a malformed block lands unscored — end-to-end through assemblePipelineStoriesclassifyPipeline.
  • OpenAPI + SDK regenerated (pnpm --filter @facility/api openapi); full typecheck and the harness/api/web suites pass locally. The one local failure, ai-identity.test.ts's vendor-asset hash, fails identically on a pristine main under a Windows CRLF checkout and is what fix: several Windows-checkout issues found while running the platform's own checks locally #173 is about.

🤖 Generated with Claude Code

https://claude.ai/code/session_018XarNuJCzYsJupYqMDUvaG

…ready records

The Stories board sorted every stage by gh_updated_at, so any webhook — a
comment, a label, Facility's own acknowledgement — reordered the board, and
the WSJF scores the PO agent writes into the issue body were never read back.

Read the ## Value section out of the mirrored issue body at assembly time,
score it with the existing wsjfScore, and sort active stages by that
judgement: scored stories first, highest score on top, unscored stories
below in GitHub's own newest-created-first order. Shipped keeps recency —
it is a log, not a queue. The score and its components ride the pipeline
and story-detail responses, and the board renders them so the ordering is
inspectable.

The serialisation moves into @facility/harness next to the parser, so the
round trip through the GitHub mirror is pinned by one pair of functions.

Closes theam#109

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018XarNuJCzYsJupYqMDUvaG

@adrian-lorenzo adrian-lorenzo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution!

The ## Value block is untrusted GitHub content, but the parser currently treats any finite numbers as authoritative. It accepts values the canonical WSJF schema rejects, and values such as 1e308 produce an infinite score that the response schema rejects, allowing one issue to make the Stories endpoint return 500. Any issue author can also promote their own story by editing its body. Please share the canonical validation, ensure the computed score remains finite, bind ranked scores to trusted provenance, and add unit and API-level regressions for negative values, overflow or tiny effort, and forged blocks.

The score breakdown is only exposed through a title on a non-focusable element, so it cannot be inspected reliably with a keyboard or touch device. Please make that detail accessible and cover the interaction.

…sue body

The review on theam#176 showed the mirrored ## Value block is untrusted input:
any issue author can edit it, the parser accepted shapes the canonical WSJF
schema rejects, and finite-looking components (1e308 sums, a subnormal
effort) divide to an infinite score the Stories response schema refuses —
one forged issue turned the endpoint into a 500.

- The canonical WsjfSchema moves into wsjf.ts and is shared with the task
  frontmatter validation in chain.ts. parseWsjfValueSection and the new
  validateWsjf accept only canonical components, and wsjfScore refuses to
  return a non-finite score.
- Pipeline assembly now binds each story's rank to trusted provenance: the
  judgement Facility recorded on the po_tasks row when it mirrored the
  issue, located through that row's own gh reference, newest record first.
  The world-writable body block never scores a story.
- The board chip discloses the score breakdown through a native
  details/summary — reachable by keyboard, touch, and assistive tech —
  instead of a hover-only title.
- Regressions: harness units for negative, fractional, overflow, subnormal,
  and forged shapes; assembly units for forged bodies and junk task
  records; an API-level test that forged blocks keep /pipeline at 200,
  unscored, and outranked by the recorded judgement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H2b6d6YmprkKziLC9BGaUG
@vicente-debug

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. The overflow-to-500 path and the self-promotion vector were both real.

Addressed in 6e24dcd:

  • Canonical validation is now shared. WsjfSchema moved into wsjf.ts and is the same schema task frontmatter uses; the parser and the new validateWsjf reject anything non-canonical.
  • Scores are guaranteed finite. wsjfScore refuses non-finite results (a subnormal effort still divides to Infinity even with canonical components), and untrusted paths turn that into "unscored", never an error.
  • Ranks bind to trusted provenance. Assembly no longer reads the issue body at all: each story's judgement comes from the po_tasks row Facility wrote when it mirrored the issue, located through that row's own gh reference. Editing the body can neither promote a story nor break the endpoint.
  • The chip is now a native details/summary disclosure, reachable by keyboard, touch, and screen reader, with tests replacing the hover-only title.

Regressions cover negative and fractional components, overflow and subnormal effort, and forged blocks at unit and API level, including an issue whose forged block claims 1e308 leaving /pipeline at 200.

@adrian-lorenzo adrian-lorenzo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Thoughtful work throughout.

@adrian-lorenzo
adrian-lorenzo merged commit d71315c into theam:main Aug 28, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Order the backlog by priority, not by last activity

2 participants