Skip to content

docs: propose governed improvement experiments - #189

Draft
emezac wants to merge 2 commits into
theam:mainfrom
emezac:docs/improvement-lab-rfc
Draft

docs: propose governed improvement experiments#189
emezac wants to merge 2 commits into
theam:mainfrom
emezac:docs/improvement-lab-rfc

Conversation

@emezac

@emezac emezac commented Aug 20, 2026

Copy link
Copy Markdown

Summary

  • define the proposed Facility Improvement Lab contract for registry-backed skill and agent-contract experiments
  • separate deterministic assignment from receipt-proven exposure
  • specify human-gated start, candidate, adoption, and rollback transitions
  • define outcome, cost, integrity, and safety guardrails with explicit assessed denominators
  • scope the first vertical slice to issue-level assignment and platform evidence, leaving shadow reviewers, guards, model routing, and workflow composition as follow-ups

Why a design PR first

This is a substantial behavior change. The repository contribution rules require its contract to be agreed before implementation. This draft provides a concrete artifact for discussion without adding migrations, routes, runtime behavior, or user-facing claims about an unavailable feature.

The proposal deliberately reuses Registry, receipts, outcomes, gateway attribution, HITL, Watchtower, budgets, and audit. It does not introduce a parallel telemetry system and does not make security/accountability invariants experiment variables.

Closes #188 only when the design is accepted; until then this remains a draft discussion artifact.

Verification

  • pnpm --filter @facility/docs build
  • node guards/run.mjs
  • git diff --check

Open decisions

  • whether the first slice should support repository-lane exposure or fail closed to platform-lane runs
  • statistical methods and defaults that expose uncertainty honestly
  • approval requirements for allocation changes
  • whether Registry needs explicit active-version history for rollback proof
  • the minimum general issue metadata suitable for cohort eligibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add governed experiments for registry-backed agent improvements

1 participant