From 22ed1ad2d546590453ca0b90875981805e5e03ad Mon Sep 17 00:00:00 2001 From: Alex Rock <1675576+nynexman4464@users.noreply.github.com> Date: Tue, 1 Sep 2026 14:46:36 -0700 Subject: [PATCH] test(vibe-tests): compare established Tailwind adoption strategies The setup evaluation has one existing-app recipe. An application that already has its own reset, token vocabulary, and visual language can adopt a design system in more than one way, and those ways disagree: keep the host authoritative over typography and colour and make the design system disappear into it, which means an app-owned theme rather than a stock theme stylesheet; or keep the host's global ownership untouched and contain the design system to one new subtree, which means deliberately not importing the reset. A single recipe cannot represent both, and combining them hands an implementer contradictory instructions. Two opt-in strategy conditions, each a self-contained document, explored additively beside the existing controls. They are not product equals and the documents no longer read as though they were. Host-aligned adoption is the normal objective for an established app and runs entirely on supported public API; its cost is maintenance, since an app-owned theme has to track the host's tokens as they change. Guest containment is the exceptional mechanism, for an app whose global ownership cannot move and where scoping the design system to one nested subtree is a hard requirement; it has no fully supported path today, because it depends on suppressing a root attribute sync no public API exposes and it gives up theming for content rendered outside the contained subtree. It is therefore as much a probe of those two product gaps as a recipe. Stages are additive. Existing stage ids, cell identities, and expansion counts are unchanged, and tests assert those counts and assert that no new cell can collide with an existing one by key or id. Iterations take new condition ids rather than editing an existing one, so an earlier result is never overwritten by a later run, and each revision records the one it supersedes. What stays neutral is the measurement: aggregation reports the two strategies separately, per failure category, with no combined score and no ranking. The s5 contract gains one more intended-change allowance. s5 mandates exactly one visible astryx-dialog-trigger, and this fixture declares a guest subtree for guest design-system content, so that is where the trigger belongs and where two independent reps put it. Both grew guest-boundary from 296px to 409px, moving its height and the two geometry fields height moves, in both schemes. That is the insertion's own geometry, not host damage, so the contract now allows those three fields on that probe: height, geometry.height, and geometry.bottom. It is narrower than the shell's allowance and takes no text exemption at all, because the trigger is task-owned and the boundary's protected text already excludes it. The boundary's width, its position, its every computed style, and every other probe stay exact. A canonical browser case inserts the trigger into the guest subtree and measures the real app, and mutation proofs over that measured run show a colour, a font, the boundary's width, the boundary moving, a neighbouring probe's height, and host copy the boundary either gains or loses all still fail. No default changes, no core or package changes, and greenfield adoption is unaffected. This repository prepares fixtures, tasks, and measurement; it never launches an executor. --- internal/vibe-tests/README.md | 2 + internal/vibe-tests/setup-test/PLAN.md | 370 +++++++- .../vibe-tests/setup-test/conditions.json | 79 ++ .../setup-test/guidance/guest-contained.md | 148 +++ .../setup-test/guidance/host-aligned.md | 188 ++++ internal/vibe-tests/setup-test/matrix.json | 146 +++ internal/vibe-tests/setup-test/prompts.json | 6 + internal/vibe-tests/setup-test/run-setup.mjs | 52 +- .../setup-test/setup-aggregate.test.ts | 201 ++++ .../vibe-tests/setup-test/setup-aggregate.ts | 210 +++++ .../setup-test/setup-canonical.test.ts | 877 +++++++++++++++++- .../vibe-tests/setup-test/setup-eval.test.ts | 213 +++++ .../vibe-tests/setup-test/setup-matrix.mjs | 104 ++- .../setup-test/setup-matrix.test.mjs | 541 ++++++++++- 14 files changed, 3090 insertions(+), 47 deletions(-) create mode 100644 internal/vibe-tests/setup-test/guidance/guest-contained.md create mode 100644 internal/vibe-tests/setup-test/guidance/host-aligned.md diff --git a/internal/vibe-tests/README.md b/internal/vibe-tests/README.md index c90ceccd3c00a..d9ea60b930788 100644 --- a/internal/vibe-tests/README.md +++ b/internal/vibe-tests/README.md @@ -153,6 +153,8 @@ internal/vibe-tests/ ├── setup-test/ # Setup evaluation over the canonical fixture matrix │ ├── PLAN.md # controls, pilot stages, measures, decision rule │ ├── matrix.json # fixtures × controls × prompts × bundles × reps +│ ├── conditions.json # controls plus the opt-in established-app strategies +│ ├── guidance/ # one document per patch, including both strategies │ ├── run-setup.mjs # sandbox + task + provenance preparation only │ ├── setup-interactions.mjs # marker-driven dialog and nested-overlay opener │ ├── setup-measure.mjs # build + exact style, geometry, and overlay probes diff --git a/internal/vibe-tests/setup-test/PLAN.md b/internal/vibe-tests/setup-test/PLAN.md index 23ad5a7979b1f..d1333728c60a2 100644 --- a/internal/vibe-tests/setup-test/PLAN.md +++ b/internal/vibe-tests/setup-test/PLAN.md @@ -46,35 +46,280 @@ Each task declares stable task-owned `data-vibe-result` markers. They are measur Guidance is committed into the sandbox baseline before execution, so the resulting diff contains only executor work. -## Evaluator correctness +## Established-app strategy pilot -Five faults where the evaluator reported an unchanged host as damaged, or reported unrelated problems with the same word. None of them relaxes what the evaluator considers host damage. +Two further conditions explore strategies for an application that already has an established visual language. They are additive and opt-in: the four controls above, their evidence, and their cell identities are untouched — every pre-existing stage keeps its exact expansion count and its exact cell ids, and a strategy is reachable only from the new stages that name it — and no default, core, or greenfield behavior changes as part of this pilot. Normal greenfield Astryx adoption is exactly what it was. -**1. `hardcoded-important` read prose as code.** The check matched `/!\s*important/i` against every added line of a host source file, so an executor told — correctly, by the guidance itself — not to paper over a containment problem with `!important` failed the run for writing that reasoning down beside the workaround it chose instead. The flag is now found by parsing rather than by matching: stylesheets go through postcss, where `Declaration.important` is the parser's own answer and a comment is a `Comment` node; scripts go through the TypeScript parser, where comments are trivia and only string and template literals are examined; HTML, Vue, and Svelte are split into their `