Skip to content

Commit 7f715cd

Browse files
fix(targets): enforce native layouts and correct conformance evidence (#49)
* fix(targets): report invalid builds and reject output collisions Keep hook pointers and custom marketplace validation consistent with emitted files. Refuse same-target overwrites before writing output, and report package skill, Codex entry, and omitted hooks problems during build. * docs: make the quick start executable across targets Provide every authored file, use Copilot-compatible agent names, and supply Codex marketplace policy fields. Materialize the README blocks in CI and run the built CLI for every configured target. Clarify oracle strength, custom marketplace validation, and target migration cleanup. * test(conformance): enforce a 90-day evidence refresh cadence Fail CI when citation or vendored-schema verification dates expire, are invalid, or are in the future. Preserve the recorded verification dates and document how to recheck claims and diff pinned schemas before advancing them. * test(conformance): check evidence dates on a schedule, not on every PR The freshness checks depend on the date, not on the change under test, so running them in npm test would turn every pull request red the day a citation ages out. Run them weekly and on demand instead, and keep the pure boundary test in the suite. * fix(targets): enforce native layouts and clarify build diagnostics Claude Code and Cursor discover manifests at fixed paths, so reject custom marketplace directories before emission instead of validating an output the clients cannot load. Legacy Codex hooks pointers must name a file that actually ships. Point skill warnings at authored files, including overrides, and keep the target illustration, collision guidance, and executable quick start aligned with the output. Refresh only target citations confirmed against current sources. * docs(conformance): refresh schema provenance and flag changed claims Re-fetch both pinned schema sets and record their byte-identical upstream revisions. Keep historical dates on two contradicted install restrictions and the undocumented legacy Codex minimum rather than falsely certifying them. The scheduled check will continue to flag those claims until they are resolved. * refactor(targets): remove unused native validation argument Keep managed paths at the shared validation layer. Native target validators no longer consume them, so retain the original two-parameter contract and enforce it with a type assertion. * fix(conformance): correct install commands and legacy evidence Use documented Claude shell commands and Antigravity GitHub URL installation. Retain interactive Claude commands and a path-aware local Antigravity fallback in the notes. Cite the tagged Codex legacy parser instead of claiming the host requires a name. Refresh only these three verified citations and cover the install-info output. * test(targets): drop the validateOutput signature pin The signature is a seam other changes may legitimately extend, for example to tell a validator whether managed state exists. The behavior tests in this file already cover what the pin was protecting.
1 parent 6858bf1 commit 7f715cd

29 files changed

Lines changed: 1274 additions & 211 deletions
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Evidence freshness
2+
3+
# Fails when a target citation or vendored schema is past the re-verification
4+
# cadence in CONFORMANCE.md. It runs on a schedule rather than on pull
5+
# requests, because its result depends on the date, not on the change.
6+
on:
7+
schedule:
8+
- cron: "0 16 * * 1"
9+
workflow_dispatch:
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
freshness:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@v7
19+
20+
- name: Set up toolchain
21+
uses: jdx/mise-action@v4
22+
23+
- name: Install dependencies
24+
run: npm ci
25+
26+
- name: Check evidence dates
27+
env:
28+
PLUGINPACK_CHECK_FRESHNESS: "1"
29+
run: npx vitest run tests/evidence-freshness.test.ts

‎CLAUDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Data flows: **config → discover source → collect/render files → emit per t
3434
diff, prune, and validate all derive from it.
3535

3636
- `src/cli.ts` — commander CLI: `init`, `build`, `validate`, `diff`, `prune`,
37-
`clean`, `docs`.
37+
`clean`, `install-info`, `docs`.
3838
- `src/schema.ts` — **zod schemas are the source of truth for config types**;
3939
authoring types use `z.input`, and parsed config uses `z.infer`. The build
4040
materializes these aliases as structural declarations without a Zod type

‎CONFORMANCE.md‎

Lines changed: 67 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,9 @@ against a temp fixture via [`bintastic`](https://github.com/scalvert/bintastic).
2424

2525
- **claude** — runs `claude plugin validate --strict` against the emitted
2626
marketplace and plugin directories when the `claude` CLI is on `PATH` (skipped
27-
otherwise, e.g. CI). This is Anthropic's own validator — the same check their
28-
submission pipeline runs — so it is a genuinely external oracle that tracks
27+
otherwise). CI installs the CLI before running the suite. This is Anthropic's
28+
own validator, the same check their submission pipeline runs, so it is an
29+
external oracle that tracks
2930
upstream automatically. A structural golden assertion also pins the exact
3031
emitted shape.
3132
- **cursor** — validates the emitted `marketplace.json` and `plugin.json` against
@@ -71,14 +72,20 @@ against a temp fixture via [`bintastic`](https://github.com/scalvert/bintastic).
7172
(re-verified 2026-10-02): a repo-scoped `.agents/plugins/marketplace.json`
7273
(`{ name, interface, plugins }`, no `owner` field). The default package is a
7374
root `plugin.json` with OpenAI settings under `extensions.com.openai`.
74-
`format: "legacy"` instead emits `.codex-plugin/plugin.json`, where only
75-
`name` is required and `skills`/`hooks`/`mcpServers` are optional pointers;
76-
`validate` detects either layout per plugin. Every marketplace entry must carry
75+
`format: "legacy"` instead emits `.codex-plugin/plugin.json`.
76+
In [Codex rust-v0.146.0's legacy parser](https://github.com/openai/codex/blob/rust-v0.146.0/codex-rs/core-plugins/src/manifest.rs#L34-L287),
77+
every recognized field has a serde default; an omitted or blank `name` falls
78+
back to the plugin directory basename. `version`, `description`, `skills`,
79+
`hooks`, and `mcpServers` are optional. Pluginpack keeps a stricter rule:
80+
an explicit `name` must match the marketplace entry. `validate` detects
81+
either layout per plugin. Every marketplace entry must carry
7782
`policy.installation`, `policy.authentication`, and `category`; pluginpack has
78-
no way to infer these, so the base entry stays guess-free and `validateOutput`
79-
errors clearly if an author never supplies them via the per-plugin `entry`
80-
passthrough. An entry's `source` is a bare string only for local plugins (the
81-
only shape pluginpack itself ever emits); a `url`/`git-subdir`/`npm` source
83+
no way to infer these, so the base entry stays guess-free. Build warns and
84+
`validateOutput` errors if an author never supplies them via the per-plugin
85+
`entry` passthrough (or `clients.codex.entry` for a package client). Build
86+
also warns about skill frontmatter that package validation rejects.
87+
An entry's `source` is a bare string only for local plugins (the only shape
88+
pluginpack itself ever emits); a `url`/`git-subdir`/`npm` source
8289
added via `entry` is a structured object with an inner `source` discriminator
8390
(e.g. `{ source: "git-subdir", url, path, ref }`), validated by shape rather
8491
than requiring a local directory to exist. No published JSON Schema exists.
@@ -214,19 +221,34 @@ marketplace. Each target's snippet and citation live on its own
214221
`PluginTargetDefinition.installSnippet` in `src/targets/<name>.ts`, verified
215222
directly against product docs:
216223

217-
| Target | Snippet | Source | Verified |
218-
| ------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- | ---------- |
219-
| `claude` | `/plugin marketplace add <repo>` then `/plugin install <name>@<marketplace>` (slash-only) | <https://code.claude.com/docs/en/plugins-reference#cli-commands-reference> | 2026-07-25 |
220-
| `codex` | `codex plugin marketplace add <repo>` | <https://learn.chatgpt.com/codex/developer-commands> | 2026-07-25 |
221-
| `copilot` | `copilot plugin marketplace add <repo>` then `copilot plugin install <name>@<marketplace>` | <https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing> | 2026-07-25 |
222-
| `antigravity` | `git clone <repo> plugin-source && cd plugin-source && agy plugin install <pluginPath>` | <https://antigravity.google/docs/cli/plugins> | 2026-07-25 |
223-
| `cursor` | No CLI equivalent exists. Prints the repo URL, pasted into Dashboard → Plugins → Team Marketplaces → "Import from Repo." | <https://cursor.com/docs/plugins> | 2026-07-25 |
224-
225-
`claude`'s two-step sequence is not symmetric with `codex`/`copilot`'s shell
226-
commands: `/plugin marketplace add` is slash-only inside an active Claude Code
227-
session, with no shell equivalent — but once a marketplace is already added,
228-
`claude plugin install <name>@<marketplace>` does work as a standalone shell
229-
command, surfaced as a secondary `note`.
224+
| Target | Snippet | Source | Verified |
225+
| ------------- | ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ---------- |
226+
| `claude` | `claude plugin marketplace add <repo>` then `claude plugin install <name>@<marketplace>` | <https://code.claude.com/docs/en/plugins/cli-reference> | 2026-10-21 |
227+
| `codex` | `codex plugin marketplace add <repo>` | <https://learn.chatgpt.com/codex/developer-commands> | 2026-10-21 |
228+
| `copilot` | `copilot plugin marketplace add <repo>` then `copilot plugin install <name>@<marketplace>` | <https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing> | 2026-10-21 |
229+
| `antigravity` | `agy plugin install https://github.com/<owner>/<repo>` | <https://antigravity.google/docs/plugins?tab=cli> | 2026-10-21 |
230+
| `cursor` | Prints the repo URL, pasted into Dashboard → Plugins & MCPs → Team Marketplaces → "Import from Repo." | <https://cursor.com/docs/plugins> | 2026-10-21 |
231+
232+
The 2026-10-21 refresh corrected both install snippets. Claude's shell
233+
commands were checked against the current command reference and Claude Code
234+
2.1.284's `claude plugin --help` and `claude plugin marketplace --help`.
235+
The old `plugins-reference#cli-commands-reference` URL now redirects to the
236+
manifest reference, so the citation points to the current command reference.
237+
The interactive `/plugin marketplace add <repo>` and
238+
`/plugin install <name>@<marketplace>` forms remain documented. The latter
239+
opens the plugin details for review and scope selection rather than installing
240+
immediately; the snippet's note explains this.
241+
242+
Antigravity's current CLI docs show the exact command
243+
`agy plugin install https://github.com/<owner>/<repo>` and the local form
244+
`agy plugin install </path/to/local/plugin>`. `install-info` uses the configured
245+
repository URL. Its note retains clone-then-local-install with the generated
246+
`pluginPath`, because the docs do not specify a GitHub URL subdirectory selector.
247+
248+
The legacy Codex citation was also corrected on 2026-10-21 against the tagged
249+
OpenAI source described above. The native parser does not require an explicit
250+
`name`; pluginpack's marketplace-matching rule does. Only these three corrected
251+
citations advance their `verifiedAt` dates in this follow-up.
230252

231253
`agent-plugins` is the one target with `userConfigurable: false`. The
232254
specification defines no marketplace or install command, and installation is
@@ -235,6 +257,29 @@ repository URL:
235257
<https://code.visualstudio.com/docs/agent-customization/agent-plugins>,
236258
verified 2026-10-02).
237259

260+
## Evidence refresh cadence
261+
262+
Re-verify every citation and vendored schema at least every 90 days.
263+
The `Evidence freshness` workflow runs `tests/evidence-freshness.test.ts` every
264+
Monday and on demand. It checks every literal `verifiedAt` in `src/` and the
265+
`Verified-at` date in each vendored schema directory's `SOURCE.md`. It fails
266+
when a date is more than 90 days old, is in the future, or is invalid. The date
267+
checks do not run in `npm test`, because their result depends on the day rather
268+
than the change. Run them locally with
269+
`env PLUGINPACK_CHECK_FRESHNESS=1 npx vitest run tests/evidence-freshness.test.ts`.
270+
The tests do not fetch the network.
271+
272+
To refresh a citation, open its `documentationUrl`, check the claim against the
273+
current product docs, and run the affected target's conformance tests. Update
274+
`verifiedAt` only after that check. Correct the implementation or record a
275+
format conflict here if the upstream behavior changed.
276+
277+
To refresh a vendored schema, follow the fetch commands in its `SOURCE.md`,
278+
diff the fetched bytes against the pinned copy, and run
279+
`npx vitest run tests/conformance.test.ts` and the freshness command above.
280+
Record the checked commit and update `Verified-at: YYYY-MM-DD` in `SOURCE.md`,
281+
even when the bytes are unchanged. Do not advance dates just to pass CI.
282+
238283
## Refreshing vendored schemas
239284

240285
The Cursor and Agent Plugins schemas are pinned copies. To update them, re-fetch

0 commit comments

Comments
 (0)