diff --git a/.drive/CODE-REVIEW.md b/.drive/CODE-REVIEW.md index eca390f35..df62092ff 100644 --- a/.drive/CODE-REVIEW.md +++ b/.drive/CODE-REVIEW.md @@ -1,24 +1,53 @@ # Code review context -Project-specific review focus areas for this repo. Reviewers load this file -before reviewing a diff. +Project-specific review questions for this repo. Both review passes load this file before reviewing a diff, and answer every question below for every file in the diff. -## Jargon sweep (every review) +## Cover the whole diff -Sweep every piece of human-readable text in the diff — comments, test -descriptions, error messages, log output, doc comments — for jargon and -overlong comments. Implementer models (Sonnet and Opus are the worst -offenders) write contracted, highly technical, incomprehensible jargon all -over the place, and especially write extremely long comments. +Review every changed file, not only the product code under `packages/`. CI workflows and actions under `.github/`, scripts under `scripts/`, examples under `examples/`, docs, ADRs, the glossary, skills and the PR title and description all count. A review that read only the source files has not reviewed the PR. -Flag and rewrite: +For each file, ask: -- Invented or coined terms where a plain description of what happens works. -- Contracted, telegraphic phrasing that has to be reread to be understood. -- Long comments. A comment should state only what the code cannot express; - most should be deleted or replaced with clearer code. -- Test descriptions written in implementation vocabulary instead of plain - English describing the observable behavior. +- Did I read every added and changed line, and every unchanged line this PR made wrong? +- If this PR edits a line that already held a shortcut or a bad comment, did I flag it? A PR that touches a line owns it. -The standard: a reader outside this session understands the text on the -first read. +## Comments + +Implementer models write long comments that explain, justify or narrate. Most should not exist. + +- Does any comment explain or justify a shortcut? Flag it, and say what the proper code is. The fix is the code, not a better comment. +- Does any comment tell history ("used to", "before this change", "was renamed"), restate what the code says, or narrate a decision? Delete it. +- Does any comment run longer than two lines? Ask what the code cannot express, keep only that, and delete the rest. +- Is a doc comment on something that is not a user-facing interface? Delete it. On a user-facing interface, is it terse? + +## Shortcuts + +- Is one value hard-coded in two places, such as the app and CI, and kept in step by a comment? Generate it in one place and pass it, or prove the behaviour without a shared constant. +- Does a test or a CI check compare against a constant it also wrote, so it proves nothing about the behaviour? It should exercise the real behaviour, such as signing a challenge with a key instead of comparing the key to a known string. +- Is a special case, workaround or "temporary" path explained by a comment instead of removed or recorded in the project's `deferred.md`? + +## CI, workflows and scripts + +- Does any `run:` step in `.github/` hold more than one command, or any shell logic: captured output, exit-code checks, `if`, loops, `grep`? Move it into a script under `scripts/` and make the step one line. +- Does a script print a secret, a token or a value it was given? It must not. +- Does every new script follow the existing ones in `scripts/`: TypeScript run with `bun`, a short usage line, plain error messages? + +## Wording + +- Is every human-readable string plain English that a reader outside the team understands on the first read? That covers comments, error messages, log lines, test names, docs and the PR text. + - Flag invented or coined terms where a plain description of what happens works. + - Flag contracted, telegraphic phrasing that has to be read twice. + - Flag the banned words: "load-bearing", "gate", "smoking gun", "belt and suspenders". +- Does a test name describe the observable behaviour, not the implementation? +- Does any record, guide, comment or PR text state a current choice as a permanent rule, with "never" and no reason? State what is decided, why, and whether it is expected to change. Keep "never" for real invariants, such as a secret never appearing in an error. +- Is any markdown prose hard-wrapped? One paragraph is one line. + +## The PR title and description + +Read them as a teammate with none of the author's context. + +- Does the title say what changes for a user, not name a mechanism? +- Does the description open with a concrete example, such as code and the output it gives? +- Does it state the decision next, then build the explanation step by step? +- Does it end with the alternatives considered, rather than lead with them? +- Does every claim in it match the code? diff --git a/.drive/projects/platform-variables/deferred.md b/.drive/projects/platform-variables/deferred.md index 06d99d435..601e70cf4 100644 --- a/.drive/projects/platform-variables/deferred.md +++ b/.drive/projects/platform-variables/deferred.md @@ -9,17 +9,24 @@ Each needs a Management API change. None blocks this project (design section 3). - **A platform record of which variables each service reads.** Composer's input documents already name them; the target would also send them to the platform, which could then deliver only those. Until then every deployment receives every row of its class. - **A service scope.** Today a name resolves at two scopes, project class and branch, and preflight lists both. Will expects the platform to add a service scope. When it does, preflight's existence check gains one list call per reader; nothing else in the deploy changes, because the node never says where its value lives and preflight is the only part of the deploy that knows which scopes exist. `prisma dev` supplies one value per name from local files, so reproducing two readers with two values locally needs a per-reader local source, decided when the scope exists. - **A void value.** Let an operator set a variable to "deliberately nothing", so the deploy report stops listing an intended absence as not set. Today the platform refuses an empty value. -- **Workspace variables and rules.** Values shared across projects, with rules for which branches see them. Composer creates no values, so this needs nothing from Composer. +- **Workspace variables and rules.** Values shared across projects, with rules for which branches see them. Composer does not create platform variable values today, so this needs nothing from Composer. - **A topology entry for a platform variable node.** The platform's application topology has a fixed list of node kinds, owned by the Portal team. The node is not sent to it until the platform has a kind for it. ## Later Composer work +- **Destroying production deletes the operator's platform variables.** Destroying an app's production environment deletes its Prisma Cloud project, and with it every variable the operator set; the platform never returns a value, so an operator who kept no copy must make a new secret. Options: keep the project when it holds rows that are not Composer's, or warn before deleting. Waits because destroy's behaviour is not this slice's and needs its own decision. - **An author-declared per-reader variable,** one that maps to the service scope the platform is expected to add, so that two readers of one name are guaranteed different values. Not built: where a value lives is the operator's choice on the platform, and nobody has a case that needs the author to force it. Wait for a real case. - **A warning when the app names a variable another module expects under a different name.** Not needed: the deploy report lists every name with its readers, so a module wired to the wrong name shows up there. - **An edge kind for "a service reads a platform variable".** Core's edges are dependencies (ADR-0063 names two relationships: depends on, exposes). The reads relation is recorded as a list beside each service's binding, not as an edge, because adding an edge kind changes core's graph types and the platform's edge families at once. Reason to wait: nothing reads the relation as an edge yet; the deploy report and errors read the list. - **`schema` on a platform secret.** A secret box whose content is not text is undefined in this design and in module-input's. A `secret(inner)` field already checks a secret's text at boot. Needs a generic box type first. - **Removing `envSecret()` and `envParam()`** after their deprecation. -- **A seeding command** that copies values from local files to the platform. Shelved by Will on 2026-11-05. +- **An explicit way to set platform variables through `deploy`.** The operator names the values and the environments they go to, and the deploy writes those rows. Expected later. Not now: it needs its own design, and an implicit copy of local values is what slice 2 removed. - **Isolating a service's environment on Prisma Cloud** without a platform record of who reads what, for example by having boot remove variables a service does not read. Not planned: it would hide variables an app reads directly from `process.env`. - **Reading a deployment's logs from `prisma deploy`.** Composer does not tail deployment logs, so a boot error on the platform is read with `prisma service logs `. The guide says so; tailing is a later change. - **A value given by a resource that also provides a contract.** `provision(resource)` flattens a contract onto the returned object, so a `value` member would collide. Allowed only on a resource with no contract until a second case needs both. +- **A separate export for `envParam()` in per-key params.** One function serves both forms today, so its `@deprecated` tag strikes through the per-key form too, which is not deprecated. A second export, with the deprecation only on the binding form, gives a clean signal. Deferred from slice 2's code review (F03): the replacement for the binding form arrives with slice 3. +- **Renaming `absentSecrets`** in the serializer and the dev payload to say unset variables. In slice 2 every document marker is an `envSecret()`, so the name is accurate. Slice 3 adds markers for plain variables, and the rename belongs there. +- **A report channel for preflight checks that are not about platform variables.** `PreflightInput.report` carries platform-variable findings and a count only. Designing a general channel waits for a second preflight check that needs one. +- **The CI script's deep imports from source paths.** `scripts/e2e-first-deploy.ts` imports from `packages/1-prisma-cloud/0-lowering/lowering/src/`, as four other scripts already do. Changing it is a change to that existing practice, not to this project. +- **The platform variables with no value in the `--report` run report.** The printed deploy report and the JSON output carry the list; the run report file holds steps, timings and the failure cause, and does not. Found in slice 2 QA. Reason to wait: no CI consumer reads the list from that file yet. +- **The one-meaning-per-name check misses an `envParam()` passed indirectly.** The check sees `envParam()` only in a service binding or in per-key params. An `envParam()` passed through a module's input or a `resolve()` dependency becomes text at Load, so a clash with a root input field of the same name is not caught. Fixing it needs Load to keep the source's name and location, which is a core change outside slice 2. Found in PR #426 review. diff --git a/.drive/projects/platform-variables/design.md b/.drive/projects/platform-variables/design.md index 1b16bb068..bcdd1ed6a 100644 --- a/.drive/projects/platform-variables/design.md +++ b/.drive/projects/platform-variables/design.md @@ -36,7 +36,7 @@ export default appRoot('shop', {}, ({ provision }) => { - `EMAIL_DELIVERY_CREDENTIAL` is a secret. The service receives it in a secret box. - `SMTP_PORT` holds the text `587` on the platform. `schema` says the text is a number, so the service's `smtpPort` field, typed `number`, receives `587`. -The deploy never reads, writes or needs any of these values. A missing value fails the deploy only when a field that must have a value reads it; otherwise the deploy lists the variable as not set and succeeds. The values are set on the platform. +The deploy does not copy any of these values from the developer's local environment, and does not read or need them. A missing value fails the deploy only when a field that must have a value reads it; otherwise the deploy lists the variable as not set and succeeds. For now the operator sets the values on the platform. The two shortcuts become spellings of the same thing. `envSecret('NAME')` is a platform secret written inline, with the meaning of `variable.secret({ name: 'NAME' })` provisioned in the root, and follows every rule in this document. `envParam('NAME')` in an input binding is a deploy-time value from the app configuration, not a platform variable, and is deprecated with `envSecret()`; a root input field replaces it. Section 12 has the detail. @@ -135,7 +135,7 @@ This reverses the glossary's statement that "there is no separate root construct ### What the node is -- **The node provisions nothing and owns nothing on the platform.** It names a variable. Its descriptor writes no row, creates no Alchemy resource, and is not written to the platform's application topology. The variable's row is the operator's, and Composer never writes on it: not a value, not a `logicalId`, nothing. The node is the app's reference to something managed elsewhere, as a Terraform data source is. +- **The node provisions nothing and owns nothing on the platform.** It names a variable. Its descriptor writes no row, creates no Alchemy resource, and is not written to the platform's application topology. The variable's row is the operator's, and Composer does not write on it: not a value, not a `logicalId`, nothing. The node is the app's reference to something managed elsewhere, as a Terraform data source is. - **The variable's identity is its name; the node's identity is its address.** `name` is a property of the node. The node's address is the path of provision ids from the root, as for every node. - **One declaration per name.** Inside the root, two `variable()` calls with one `name` fail Load, naming both ids; so do a plain and a secret with one name. This cannot be structural, because the two calls are two lines in the root's body, but a collision is always between two lines the app's author wrote and can fix. The same node object provisioned twice is two declarations of one name, and fails the same way. - **`provision()` returns the value itself.** Provisioning the node returns a runtime value with `id` on it, not a wrapper: `const from = provision(variable({ name: 'EMAIL_FROM' }))` gives a `RuntimeValue & { id: string }`, which a binding uses directly. The precedent is `generatedParam()` and `serviceOrigin()`, which each return one object that is both source and value. The pointer names the variable: a plain pointer for `variable()`, a secret pointer for `variable.secret()`. @@ -195,6 +195,8 @@ A platform variable's name must match `^[A-Z][A-Z0-9_]*$`: it starts with an upp Upper case only, because the platform accepts no other key (section 3), and because the two places a name is used do not agree about case: Composer compares pointer names without case, and a process environment on Linux compares them with case. With upper-case names a row and a name can never differ by case, so boot always finds the row preflight saw. +No leading underscore, though the platform accepts one. Shells and tools use names that start with an underscore for their own variables: bash puts `_` in the environment of every command it runs, and the Java virtual machine reads `_JAVA_OPTIONS`. The rule is Composer's, and the error says so. + ### Reserved names The constructor refuses these names, saying why: @@ -305,7 +307,7 @@ An empty value cannot exist on the platform (section 3), so at boot a variable i ## 9. Deploy -- **The deploy writes no variable.** Preflight no longer creates a missing variable from the app secrets or the app configuration. That was seeding, and with rows per class and per branch, seeding as a side effect of `deploy` is how a developer's local test key reaches production. A variable is set on the platform. +- **The deploy does not copy local values to platform variables.** Preflight no longer creates a missing variable from the app secrets or the app configuration. An implicit copy can push a local value into a remote environment by accident: `.env` is read for every environment, and copying on each deploy also writes one override row per preview branch. For now the operator sets each value on the platform with `prisma project env add`, and preflight only checks that what a service needs is there. This is not a rule that the deploy may never set a variable: an explicit way to set variables through `deploy`, where the operator says which values go to which environment, is expected later (deferred.md). - **Preflight reads existence, never a value.** For each platform variable the graph reads, preflight asks whether a row exists at any scope one of its readers resolves from, following the platform's own resolution: the environment's project class and the environment's branch, the two scopes the platform offers today (section 3). A row at either counts as existing, and preflight keeps the `updatedAt` of the row the platform resolves: the branch row when there is one, otherwise the class row. A change to the class row while a branch override exists does not change what the service receives, so it is not a trigger. The reads run in parallel, at most eight at a time; a failed list call still fails the deploy, because the triggers need `updatedAt`. Preflight does not list every row of the class, because in the preview class that returns every branch's `COMPOSER_` rows. When the platform adds a service scope, preflight gains one list call per reader, and nothing else in the deploy changes. - **A missing variable that a service needs fails the deploy.** A deployment is started and promoted as soon as it is created, and the platform reports a process that exits at boot as running (section 3), so a service that exits at boot is promoted while crashing and the deploy succeeds. Preflight prevents the case it can know about: for each service, it first runs the service's schema with no variable left out, and a failure there is an invalid input, reported as such and never blamed on a variable. Then it resolves the input binding with every variable that has no row left out and runs the schema again. If the schema passes, those variables are optional. If it fails, preflight checks each one alone and names as required the variables whose absence alone fails; when no single one fails alone, it says the service needs some of them. A marker in an array position is required without a check. This is the method ADR-0058 gives the dev preflight, and it never needs a value. The error lists each required variable, the service and field that read it, and the command that sets it. - **A missing variable nothing needs is reported.** The deploy succeeds, and the variable is listed under "Platform variables with no value", with each service that reads it and the command that sets it. @@ -374,7 +376,7 @@ The graph is still a composition of modules that are all the same, except the ro - **Glossary, every "implicit root Module" passage** (the two Aggregate Contract passages and the superseded-terms arrow). Now "app root". - **Glossary, new entry App root:** the outermost module, the one the entry default-exports and nothing provisions, built with `appRoot()`. Also called the root, the root module or the root node; those terms stay where they are exact and point at this entry. -### Slice 2: the deploy fills nothing +### Slice 2: the deploy stops copying local values - **Glossary, The app secrets, Secrets key, Named secrets, Known secrets:** each defines its set through `envSecret()` and says "Deploy preflight fills a missing `envSecret()` platform variable". Now the app secrets fill nothing on the platform, and each set is defined through the names the graph's secret sources use. Slice 3 redefines them through platform secrets. - **Glossary, The app configuration:** "Deploy preflight also fills a missing platform variable" is gone. Under `prisma dev` the local target fills a plain platform variable from it (slice 3 adds that sentence). @@ -384,11 +386,11 @@ The graph is still a composition of modules that are all the same, except the ro - **ADR-0042, service input:** the `$secret` marker; "Every variable a service's markers name must be distinct once normalised"; the three boot outcomes. Now the `$platform` marker, distinct names for `$generated` only, the four boot steps, the empty-text rule, and the array-position rule. - **ADR-0058, dev secrets:** "Each service gets only the variables its own rows point at" is stated as a local-target rule, not a Prisma Cloud one; and the recorded difference between dev's and deploy's input documents for an unset optional secret closes, because both now leave the key out. - **ADR-0062, values and config adapters:** "Deploy preflight and dev preflight fill a missing `envSecret()` platform variable". Now preflight fills nothing. Slice 3 rewrites the Values section. -- **ADR-0064, new, "A platform variable is a node":** created with the platform facts of section 3, the graph check hook, the preflight findings, and sections 8 to 12 as they apply to every operator variable. Slice 3 adds the node. +- **ADR-0064, new, "The deploy does not copy local values to platform variables":** created with the platform facts of section 3, the graph check hook, the preflight findings, and sections 8 to 12 as they apply to every operator variable. Slice 3 adds the node. ### Slice 3: a platform variable is a node -- **Glossary, Core nouns:** "Anything without a lifecycle — an API key, an external URL — is part of a node's Input, not a node." Now a node either has a lifecycle Composer manages, or refers to something the platform owns; Composer lists a referenced value and never creates it. +- **Glossary, Core nouns:** "Anything without a lifecycle — an API key, an external URL — is part of a node's Input, not a node." Now a node either has a lifecycle Composer manages, or refers to something the platform owns; Composer lists a referenced value and does not create it. - **Glossary, Resource:** "A node whose lifecycle a provider manages." Now also a node that refers to a value the platform owns. - **Architectural principles, "A node's address is its identity, everywhere," and ADR-0051:** a node's one identity is its address, written as the `logicalId` of its platform row. Now a platform variable node, which has no row and names something Composer does not own, is identified by the variable's name; the principle gains that one exception, with its reason. - **Glossary, Topology:** "Every node in the topology has a managed lifecycle." Now every node either has one or refers to something the operator sets, and the second kind is listed by name when an environment is created. @@ -430,7 +432,7 @@ No two slices rewrite one record at the same time: each slice starts after the o - **`schema` on a platform secret.** Deferred: a secret box whose content is not text is undefined in both this design and module-input's; `secret(inner)` already checks a secret's text at boot. - **Run the module's schema at boot.** Impossible: the schema is code in the app's module, which is not in the service's bundle. - **Let the platform validate.** Will expects the platform never to validate values against a schema. Composer validates at boot and in `prisma dev`. -- **Keep seeding on deploy, or add a seeding command.** Seeding on deploy is how a local value reaches production. A seeding command was shelved by Will on 2026-11-05. +- **Keep copying local values on deploy.** An implicit copy is how a local value reaches a remote environment by accident. An explicit way to set variables through `deploy` is expected later (deferred.md). - **Make `envSecret()` a node provisioned where it is written.** Rejected: Load would have to create a node from a value found in a binding, with an owner and an id nobody has defined, for a form the same slice deprecates. - **"Subscription" as the word for a service reading a variable.** Rejected: it names a platform mechanism that does not exist, reads as "receives updates as they happen", which is false, and already means streaming in the principles. - **Wait for the platform to record which service reads which variable.** Not needed: every part ships against today's API (section 3), and the platform's record would change nothing an author writes. diff --git a/.drive/projects/platform-variables/plan.md b/.drive/projects/platform-variables/plan.md index 838dd8902..6c70ad624 100644 --- a/.drive/projects/platform-variables/plan.md +++ b/.drive/projects/platform-variables/plan.md @@ -10,7 +10,7 @@ Written from scratch on 2026-11-06 against `main` after #419 plus this branch's ```text 1. The root is written with appRoot() -2. The deploy fills nothing +2. The deploy stops copying local values 3. A platform variable is a node 4. A platform variable has a schema ``` @@ -40,11 +40,11 @@ Each slice is one pull request, merged before the next starts. Slice 1 stands al **Spec:** [slices/1-the-root-is-written-with-approot/spec.md](slices/1-the-root-is-written-with-approot/spec.md). -## 2. The deploy fills nothing +## 2. The deploy stops copying local values **Starts from:** slice 1 merged. -**After this slice merges:** `prisma deploy` never creates a variable on the platform. Preflight reads only whether each variable an app names has a row, at the two scopes the platform offers (the environment's project class and its branch), and when it last changed. A variable with no row that a service needs, decided from the service's schema and never from a value, fails the deploy before anything is created, naming the variable, the service and field that read it, and the `prisma project env add` command that sets it. A variable with no row that nothing needs is listed after the preflight step and again in the deploy report, and the deploy succeeds. A changed value redeploys exactly the services that read it. An `envSecret()` or `envParam()` name must be upper case and must not be `COMPOSER_…`, `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` or `HOME`; one name has one meaning in an app. A service's input document carries `{ "$platform": { "name": "X", "secret": true } }` where it carried `{ "$secret": "X" }`. At boot a variable with no value leaves its key out and the service's schema decides; an empty value is unset; a boot error names the variable and never its text. `prisma dev` gives a service only the variables its markers and rows point at, applies the same missing-value rule, and stops before any service starts when a required one is unset. `envParam()` in a binding is deprecated. There is no node yet: `envSecret()` is the only way to name a platform secret, and every value is text. +**After this slice merges:** `prisma deploy` does not copy local values to the platform, and creates no platform variable. Preflight reads only whether each variable an app names has a row, at the two scopes the platform offers (the environment's project class and its branch), and when it last changed. A variable with no row that a service needs, decided from the service's schema and never from a value, fails the deploy before anything is created, naming the variable, the service and field that read it, and the `prisma project env add` command that sets it. A variable with no row that nothing needs is listed after the preflight step and again in the deploy report, and the deploy succeeds. A changed value redeploys exactly the services that read it. An `envSecret()` or `envParam()` name must be upper case and must not be `COMPOSER_…`, `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` or `HOME`; one name has one meaning in an app. A service's input document carries `{ "$platform": { "name": "X", "secret": true } }` where it carried `{ "$secret": "X" }`. At boot a variable with no value leaves its key out and the service's schema decides; an empty value is unset; a boot error names the variable and never its text. `prisma dev` gives a service only the variables its markers and rows point at, applies the same missing-value rule, and stops before any service starts when a required one is unset. `envParam()` in a binding is deprecated. There is no node yet: `envSecret()` is the only way to name a platform secret, and every value is text. **Satisfies:** R2, R6, R7; R5 for everything that exists in this slice. @@ -58,7 +58,7 @@ Each slice is one pull request, merged before the next starts. Slice 1 stands al - Boot: the four steps of design section 8, with "decode" meaning "the value is the text" until slice 4; a marker in an array position fails boot when its variable has no value; an empty value is unset; an error names the variable and never holds the text. An unset `envParam()` per-key param takes its default or is left out when optional, and the error for a required one says to set the variable on the platform (design section 12). - Preflight: fills nothing; reads existence at the project class and the branch, in parallel and at most eight at a time, and a failed list call fails the deploy; refuses a row marked `isManagedBySystem`; decides "required" by resolving each service's binding with every unset variable left out and running its schema, then each alone, as ADR-0058's dev preflight does; a marker in an array position is required without a check; reports the optional ones with `prisma project env add 'NAME=' --project '' --role production`, or for every other environment the `--role preview` form first and the `--branch` form after it; each service's deployment triggers carry `:updatedAt`, or `?` for no row (design section 9). - The shared package: one module, shipped in the service runtime and free of Node and Bun, holding the marker format, the decoding rule (text only for now) and the error text. Boot, `prisma dev`'s first process and the local target import it; the local target's copies of the marker constants go (design section 7, "One implementation"). -- `prisma dev`: the local target collects `$platform` markers at any depth for isolation; fills each platform secret from the app secrets under its name and hands it to the second process in memory; the named secrets and known secrets are the values under the names the graph's `$platform` markers carry; the missing-value rule of ADR-0058 leaves every unset variable out and validates, so dev's and deploy's input documents agree for an unset optional secret; an empty value is unset; a changed value restarts its readers (design section 10). +- `prisma dev`: the local target collects `$platform` markers at any depth for isolation; fills each platform secret from the app secrets under its name and hands it to the second process in memory; the named secrets and known secrets stay the values under the names the graph's secret sources use, which is every marker's name and any secret a module's input names that no service reads; the missing-value rule of ADR-0058 leaves every unset variable out and validates, so dev's and deploy's input documents agree for an unset optional secret; an empty value is unset; a changed value restarts its readers (design section 10). - `envParam()` in a binding is marked deprecated, with a message that a root input field replaces it. - Records, each rewritten in place as the current statement: ADR-0064 written, holding the platform facts of design section 3 and the rules of sections 8 to 12 as they apply to every operator variable; ADR-0029 and ADR-0062 (preflight fills nothing); ADR-0032 (an unset pointed param takes its default); ADR-0042 (the `$platform` marker, distinct names for `$generated` only, the four boot steps, the array-position rule); ADR-0058 (isolation as a local-target rule, the closed dev-versus-deploy difference); the glossary's The app configuration, The app secrets, Secrets key, Named secrets and Known secrets (nothing fills a platform variable; the sets are defined through the names the graph's secret sources use); the purpose doc's goals (a new environment needs its variables set first, and preflight names them). Design section 14 lists the statements each one reverses. - The guide: the deploy fills nothing; a required variable with no row refuses the deploy and an optional one is reported; the commands per scope; and one sentence that a service that fails at boot on the platform is still reported running and promoted, with `prisma service logs ` as where the error is. @@ -66,7 +66,7 @@ Each slice is one pull request, merged before the next starts. Slice 1 stands al **Checked on Prisma Cloud:** an `envSecret()` a service requires, unset: preflight refuses, naming it, before anything is created. One nothing requires, unset: the deploy succeeds and the report lists it with the command. Set, then changed: only its readers redeploy. The platform's rows listed before and after a deploy, showing nothing written. A fresh preview branch deploying with a preview-wide row made with `--role preview`. A secret planted in a value, and the deploy output, the report, the stack file and a boot error searched for any part of it. Under `prisma dev`: a required secret unset stops dev before any service starts; a service receives only the variables its markers name. -**Spec:** to be written when slice 1 merges. +**Spec:** [slices/2-the-deploy-fills-nothing/spec.md](slices/2-the-deploy-fills-nothing/spec.md). ## 3. A platform variable is a node diff --git a/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/plan.md b/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/plan.md new file mode 100644 index 000000000..36edb679e --- /dev/null +++ b/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/plan.md @@ -0,0 +1,47 @@ +# Slice 2 dispatch plan + +Spec: [spec.md](spec.md). Five dispatches, in order. Each lands as one or more commits on `claude/platform-variables-s2` with its tests green; the next starts from that state. + +## D1. The input document carries `$platform`, and boot reads it + +**Outcome:** one package holds the marker format, and the serializer, boot and the local target all use it. Spec 2.3, 2.4, 2.5. + +- Builds on: `main`. +- Delivers: `@internal/input-document` with its architecture entry; the serializer writes `$platform` and drops `$secret`; distinct names for `$generated` only; `InputDocumentRow.variables`; the compute descriptor's triggers read it and its `input` detail prints `$platform NAME`; boot's four steps, empty means unset, the per-key param rule and its new error; the local target imports the shared package and its copies go; the invariants test's file list updated. +- Hands to: D3 (preflight reads `variables` and the marker walk) and D4 (the report detail). +- Completed when: the target, local-target and CLI test suites pass; `dev-pointed-variables.test.ts` proves both sides agree; a planted-secret boot test passes. + +## D2. Names, and the graph check hook + +**Outcome:** a bad name fails Load with one checker's message, and an extension can refuse a graph right after Load in every command that runs Load. Spec 2.1, 2.2. + +- Builds on: D1 (nothing structural; sequential to avoid conflicts in `secret.ts`/`param.ts` tests). +- Delivers: `checkGraph` on `ExtensionDescriptor`, called in `runPipeline` after Load and before coverage, surfaced as `COMPOSE.GRAPH_INVALID`; `assertVariableName` in the target, used by `envSecret()` and `envParam()`; the target's `checkGraph` refusing `envParam()` plus a root input field of one name; the seeding-era preflight test that builds a graph Load refuses removed. +- Hands to: D3 (the checker is reused by nothing in slice 2, but preflight's names are now known valid). +- Completed when: core, CLI and target tests pass; a pipeline test shows the hook runs for deploy, dev and destroy and not for `resolveAppIdentity`. + +## D3. Preflight reads existence and decides required or reported + +**Outcome:** `prisma deploy` writes no variable, refuses only a required one, and returns findings for the rest. Spec 2.6, target side. + +- Builds on: D1 (`variables`, the marker walk) and D2 (valid names). +- Delivers: `fillMissing` and its tests gone; one list call per name, at most eight at a time; the branch row over the class row for `updatedAt`; `isManagedBySystem` refused; the ADR-0058 method moved where both preflights use it and applied per service, with the array-position and per-key rules; module-input secrets no service reads reported, never required; the required error text; the findings value returned beside the `updatedAt` map and serialised on the preflight transport. +- Hands to: D4 (the findings type and the transport payload). +- Completed when: `preflight.test.ts` and `preflight-payload.test.ts` pass with the new assertions; a test shows `POST /v1/environment-variables` is never called. + +## D4. The findings reach the operator, and `prisma dev` follows the same rules + +**Outcome:** the operator sees every variable with no value after the preflight step and in the deploy report, and `prisma dev` applies the per-key rule. Spec 2.6 CLI side, 2.7, 2.8. + +- Builds on: D3 (the findings type). +- Delivers: `PreflightInput.report` carries findings; a new CLI event the human renderer prints as the `Platform variables with no value:` block and the JSON renderer carries; step data keeps `checked` and `unset`; the second process repeats the block at the end of the deploy report; the dev preflight's per-key param rule; `@deprecated` on `envParam()`. +- Hands to: D5. +- Completed when: `operations.test.ts`, `deploy.test.ts`, `local-target-preflight.test.ts` pass; the "an app secrets value appears in no event, result or report" test still passes with the findings in the events. + +## D5. Records, guides and the release note + +**Outcome:** every record matches the code. Spec 2.9, 2.10. + +- Builds on: D1 to D4. +- Delivers: ADR-0064 and its index line; ADR-0029, 0032, 0042, 0058, 0062 rewritten in place; the five glossary entries; `goals.md`; the doc comments in `app-config.ts`; `deploying.md`, `building-an-app.md`, `deploy-cli.md`, the core-concepts skill; the release note drafted at `wip/pv-review/pr-s2-body.md`. +- Completed when: `pnpm lint` passes and a grep for "preflight fills", "fills a missing" and `$secret` over `docs/`, `skills/` and source doc comments finds only the release note's before/after. diff --git a/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/spec.md b/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/spec.md new file mode 100644 index 000000000..5db1982d7 --- /dev/null +++ b/.drive/projects/platform-variables/slices/2-the-deploy-fills-nothing/spec.md @@ -0,0 +1,188 @@ +# 2. The deploy stops copying local values + +Parent: [project spec](../../spec.md), [plan](../../plan.md) section 2, [design](../../design.md) sections 3, 5, 6, 8 to 13. Grounding against the code on 2026-11-06: `wip/pv-review/slice2-grounding.md`. + +## 1. At a glance + +```ts +// module.ts +export default appRoot('shop', ({ provision }) => { + provision(emailService(), { + input: { from: 'shop@example.com', apiKey: envSecret('EMAIL_API_KEY') }, + }); +}); +``` + +```sh +$ prisma deploy +✔ check environment variables +Platform variables with no value: + - EMAIL_API_KEY (read by service "shop.email", field "apiKey") + prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview + prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview --branch 'staging' +``` + +- The deploy does not copy local values to platform variables, because an implicit copy can push a local value into a remote environment by accident. An explicit way to set variables through `deploy` is expected later (deferred.md). Today preflight copies a missing `envSecret()` value from the developer's `.env` to the platform; after this slice it only reads whether a row exists. +- A variable with no row fails the deploy only when a service cannot boot without it, decided from the service's schema and never from a value. One nothing needs is listed, after the preflight step and again in the deploy report, with the command that sets it. The deploy succeeds. +- Every operator variable that exists today, `envSecret()` and `envParam()`, follows one rule set: upper-case names, reserved names, one meaning per name, a `$platform` marker in the input document, four boot steps, existence at both scopes the platform offers, triggers on the resolved row's `updatedAt`, and the same rules under `prisma dev`. +- There is no node yet. `envSecret()` is the only way to name a platform secret, and every value is text. Slice 3 adds the node under these rules, so no platform row ever follows two rule sets on `main`. + +This slice is one review because it is one rule change: every rule here is about the same thing, what Composer does with an operator's variable, and shipping half of them would leave two rule sets live at once. It is breaking, and the release note says how to move. + +## 2. Chosen design + +Each subsection names the design section it builds and the engineering choices made here. A choice is marked **(decided here)** where the plan left it open or the code made the plan's wording wrong. + +### 2.1 The graph check hook + +Design section 5. Core's `ExtensionDescriptor` gains an optional `checkGraph(graph: Graph): void`. The CLI's `runPipeline` calls every extension's `checkGraph` right after Load and before registry coverage, so it runs in `prisma deploy`, `prisma dev` (first start and every rebuild) and `destroy`, and never in `log`, which does not run Load. The hook refuses a graph by throwing core's `GraphCheckError`, which fails the command as `COMPOSE.GRAPH_INVALID` with its message; nothing is created and no dev service starts. Any other throw is a bug (ADR-0044). + +**(decided here)** The hook takes the loaded graph only. It has no access to the app configuration or secrets: a graph check is about the graph. + +The Prisma Cloud extension registers one `checkGraph`. In this slice it enforces one meaning per name (2.2). Slice 3 adds the node rules. + +### 2.2 Names + +Design section 6. One checker in the Prisma Cloud target, `assertVariableName(name, factory)`, refuses a name that does not match `^[A-Z][A-Z0-9_]*$` and the reserved names `COMPOSER_…`, `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` and `HOME`, each with its reason from design section 6. `envSecret()` and `envParam()` call it at construction and lose their duplicated copies, so Load fails with the checker's message. Slice 3's constructor reuses it. The messages follow today's shape, for example: + +```text +envSecret name "api_key" is not a platform variable name: it must match ^[A-Z][A-Z0-9_]*$, an upper-case letter followed by upper-case letters, digits and underscores. The platform accepts only upper-case letters, digits and underscores in a key. Refusing a leading underscore, which the platform accepts, is Composer's own rule: shells and tools use names that start with an underscore for their own variables. +envSecret name "PORT" is reserved: boot sets PORT to the service's port. +``` + +**One meaning per name.** Core's Load already refuses a name read from the app configuration (`envParam()` in a binding or in per-key params, or a root input field) that another place names as a secret (`envSecret()`), because Composer writes plain values to a file and never a secret. **(decided here)** That check stays in core as it is: it protects core's own file rule, and it runs before any hook. The target's `checkGraph` adds the pair core does not refuse: one name used by `envParam()` and by a root input field. The error names both uses and both addresses. Two `envParam('X')`, or two `envSecret('X')`, in two bindings read one variable and stay allowed, as today. The slice updates `preflight.test.ts`'s "a name bound as a secret and as an env-sourced param is filled from the app secrets", which builds a graph Load refuses; it goes with seeding. + +### 2.3 The shared package + +Design section 7, "One implementation". **(decided here)** A new package `@internal/input-document` at `packages/1-prisma-cloud/0-lowering/input-document`, lowering layer, shared plane, with no dependencies and no Node or Bun import, on the model of `@internal/s3-protocol`, with its `architecture.config.json` entry. It holds: + +- the marker constants and types: `$platform` with `{ name, secret? }`, `$generated`, the key escape (`^\$+(?:platform|generated)$`), and the param pointer prefix; +- `collectPlatformMarkers(document)`, which finds every `$platform` marker at any depth and returns `{ name, path, secret, inArray }`; +- the decoding rule, which in this slice is "the value is the text", and the boot error texts for a marker: a variable with no value in an array position, and the list of unset names appended to a schema failure. + +The target's serializer, the service runtime's boot code and the local target import it. The local target's copies in `local-target/src/compute.ts` go, and `dev-pointed-variables.test.ts` keeps proving the two sides agree. The target's `invariants.test.ts` list of files that read `process.env` is updated for the moved code. + +### 2.4 The input document + +Design section 8. The serializer writes `{ "$platform": { "name": "X", "secret": true } }` where it wrote `{ "$secret": "X" }`. `$secret` is gone from the writer and the reader; the document and the boot code ship in the same package, so no compatibility reader exists. The `$`-key escape covers `$platform`, `$generated` and, so that an old user key still round-trips, `$secret`. The distinct-names check applies to `$generated` markers only; two `$platform` markers for one name in one binding are two reads of one variable and are allowed, and the tests that assert a collision for secrets change to assert the allowance. `InputDocumentRow.secrets` becomes `InputDocumentRow.markers`, every `$platform` marker the document holds with its name, path, `secret` and whether an array position holds it, because the compute descriptor builds triggers from it and both preflights name readers and array positions from it. + +**(decided here)** The deploy report's `input` detail prints the document with each marker replaced by the text `$platform NAME` or `$generated NAME`, and no longer the marker's JSON. Slice 4 adds `(number)` for a typed one. + +### 2.5 Boot + +Design sections 8 and 12. `service.input()` follows four steps: + +1. A marker whose variable has no value, meaning `process.env[name]` is undefined or empty, leaves its key out of its object. A marker in an array position cannot be left out: boot fails naming the variable and the path. **(decided here)** Empty means unset for a per-key `envParam()` pointer too; today an empty value reaches the schema. The two serializer tests that assert an empty value passes through change. +2. A marker whose variable has text takes the text. Slice 4 adds decoding and validation here. +3. A marker with `secret: true` puts the text in a secret box. +4. The service's own schema validates the whole object, as today. When it fails and at least one marker was left out in step 1, the error ends with the names of those variables, so the operator knows what to set. The service's schema message stays in the error, as today: every secret is boxed before the schema runs, so the message can hold no secret's text. + +A per-key `envParam()` param whose pointed variable has no value takes the param's default, or is left out when the param is optional, and fails boot only when it is required. The error for a required one is: + +```text +config param "" (env → ) has no value: set the platform variable "" with prisma project env add. +``` + +Today it says "the deploy did not provision it", which stops being true. + +### 2.6 Preflight + +Design section 9. Preflight fills nothing: `fillMissing` and every test of it go. For each platform variable the graph names, preflight makes one list call filtered by `projectId`, `class` and `key`, which already returns the class row and the branch row, walks every page, and keeps the rows the environment sees: the class row, and for a preview environment its branch's row. **(decided here)** "Existence at both scopes" is that one call per name, not two. + +- **Resolution.** The branch row wins when there is one, otherwise the class row. The `updatedAt` preflight hands back is that row's, so a change to a class row shadowed by a branch override triggers nothing. The two tests that assert "the newer row wins even when it is the template" change to assert this. +- **A system row.** A row the environment sees that carries `isManagedBySystem: true` fails the deploy, naming the variable: such a row is the platform's whatever its name. The page projection gains that field. +- **Concurrency.** The list calls run at most eight at a time. There is no helper in the repo; the target gains a small one. A failed list call fails the deploy, as today. +- **Required or reported.** For each service, preflight first serialises the input with every variable present and runs the service's schema; a failure there is an invalid input, reported as such and never blamed on a variable. Then it serialises with every variable that has no row left out and runs the schema again; if it passes, those variables are optional for that service. If it fails, each one alone is left out in turn, and the ones whose absence alone fails are required; when no single one fails alone, the service "needs some of them". A marker in an array position whose variable has no row is required without a check. This is the local target's `requiredSecrets` method, moved where both preflights can use it. **(decided here)** A per-key `envParam()` param is required when it has no default and is not optional, which is the boot rule in 2.5. **(decided here)** A secret a module's input names that no service reads has no reader to need it: it is reported, never required. +- **The error for a required variable** lists each one with its readers and the commands, and no service or database is created. Preflight runs after the deploy has resolved the Project and Branch, so a new app's first deploy creates those two and then stops here, with the project's id in the commands: + +```text +Deploy failed. 1 platform variable a service needs has no value: + - EMAIL_API_KEY (read by service "shop.email", field "apiKey") + +Set it on the platform, then deploy again: + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview --branch 'staging' +``` + + For production the one command is `--role production`. For every other environment the preview-wide form comes first and the branch form second, as design section 9 says. + +- **The findings.** Preflight hands the CLI, through `PreflightInput.report`, the list of variables with no value that nothing requires, each with its readers (service address and field path, or the per-key param) and the commands. `PreflightInput.report` carries this list in place of counts; its doc comment "counts only: never a name or a value" becomes "names and readers only: never a value". **(decided here)** The CLI prints the list right after the preflight step as a block headed `Platform variables with no value:`, through a new event the human renderer prints and the JSON renderer carries; step data stays a flat record and keeps `checked` and `unset` counts. The CLI carries the list to the second process in core's own transport variable, and lowering puts it into `DeploymentResult.findings`, so the deploy report repeats it at the end under the same heading. The extension's own preflight payload does not carry it. `prisma dev` has its own rule (2.7) and prints nothing here. +- **Triggers.** Unchanged in shape: each service's deployment triggers carry `:updatedAt` for each variable its document or its per-key params point at, and `?` for no row. With resolution as above, changing a value redeploys exactly its readers. + +### 2.7 `prisma dev` + +Design section 10. The local target collects `$platform` markers at any depth through the shared package and gives each service only the variables its own rows and markers point at, as today. A platform secret is filled from the app secrets under its name and reaches the second process in `PRISMA_COMPOSER_NAMED_SECRETS`, in memory, as today. + +**(decided here)** The named secrets and the known secrets stay keyed by `secretsKeys(graph)`, every secrets key of the graph's secret sources. Design section 14 defines them through "the names the graph's secret sources use", and that is this set; it is a superset of the marker names only for a module-input secret no service reads, and redacting one more value is the safe side. The plan's sentence that names the markers is corrected in `plan.md`. + +The dev preflight already follows the ADR-0058 method for secrets. It changes in one place: a per-key `envParam()` param whose variable is unset fails only when the param is required (no default, not optional), with the same wording rule as the deploy's error, and its test "a missing env-sourced param is a hard error naming the variable and service" changes to that. An empty value is unset. A changed value keeps today's behaviour: dev pauses rebuilds and asks for a restart, and after the restart the fingerprint trigger redeploys only the readers. + +### 2.8 `envParam()` in a binding is deprecated + +Design section 12. The JSDoc of `envParam()` gains `@deprecated` with the message "a root input field replaces envParam() in a binding; per-key params are unchanged", so editors strike it through. **(decided here)** Nothing else marks it: the repo has no deprecation convention, and a JSDoc tag is the one every TypeScript editor shows. Its behaviour does not change in this slice beyond 2.5 and 2.6. + +### 2.9 Records + +Design section 14, slice 2. Each is rewritten in place as the current statement, with no supersession note. + +- **ADR-0064, new.** **(decided here)** Its file is `ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md`, and `design.md` section 14 is corrected to that title, because a record titled "a platform variable is a node" would be false on `main` until slice 3 merges, and the chosen title stays true after it. It records the platform facts of design section 3, the graph check hook, the name rules, the `$platform` marker, the four boot steps, the preflight rules and findings, the `prisma dev` rules and the shortcuts' status, as they apply to every operator variable. Its line goes into `docs/design/90-decisions/README.md`. +- **ADR-0029 and ADR-0062:** preflight fills nothing. **ADR-0032:** an unset pointed param takes its default or is left out when optional. **ADR-0042:** the `$platform` marker, distinct names for `$generated` only, the four boot steps, empty means unset, the array-position rule. **ADR-0058:** isolation is a local-target rule, and the input a service receives under `prisma dev` and on the platform now agrees for an unset optional secret: the key is absent. Dev's document leaves the marker out; the deploy's document holds it, and boot leaves the key out. +- **Glossary:** The app configuration, The app secrets, Secrets key, Named secrets and Known secrets say nothing fills a platform variable, and define each set through the names the graph's secret sources use. +- **Purpose, `goals.md`:** a new environment needs its platform variables set first, and preflight names every one a service needs before any service or database is created. +- **Doc comments** on `PreflightInput`, `AppSecrets` and `NamedSecrets` in `core/src/control/app-config.ts` stop saying preflight fills. +- **Guides:** `docs/guides/deploying.md` (the seeding paragraph, the step list), `docs/guides/building-an-app.md` (the `envParam`/`envSecret` section), `docs/design/10-domains/deploy-cli.md`, and `skills/prisma-composer-core-concepts/SKILL.md`. The deploy guide says in one sentence that a service that fails at boot on the platform is still reported running and promoted, and names `prisma service logs ` as where the error is. + +### 2.10 The release note + +In the PR description, a before and after for each: the one-time step first, creating a preview-wide value with `--role preview` for every name the project seeded; the deploy no longer creates a variable; it fails for a missing variable only when a service needs it; names are upper case, and `PORT`, `PATH` and `HOME` are reserved; a lower-case name, which only ever worked in `prisma dev`, is renamed; a leading-underscore name is renamed after its row is created under the new name in every scope that has one; an unset per-key `envParam()` with a default boots; empty means unset; `envParam()` in a binding is deprecated. + +## 3. Expected behaviour + +Everything below is observable without reading the code. + +- `envSecret('api_key')`, `envParam('_TOKEN')`, `envSecret('PORT')` each fail Load with a message naming the rule. `envParam('X')` in a binding and `X` as a root input field fail Load naming both. +- A service whose input holds `envSecret('X')` serialises to a document holding `{ "$platform": { "name": "X", "secret": true } }`; two such leaves in one binding are allowed; a user key `$platform` round-trips as `$$platform`. +- At boot with `X` unset: a required field fails the schema and the error ends with `X`; an optional field is absent and the service runs; an array position fails naming `X`. With `X=""`: the same as unset. With `X=abc`: the field holds a secret box of `abc`. +- A per-key `envParam()` param with a default boots with the default when its variable is unset or empty. +- `prisma deploy` with `X` unset: when a service requires it, the deploy fails after the preflight step with the error in 2.6 and the platform's rows are unchanged; when nothing requires it, the deploy succeeds, the block "Platform variables with no value" prints after the preflight step and again at the end of the report, and the rows are unchanged. Composer writes to `/v1/environment-variables` only its own `COMPOSER_` rows and the `DATABASE_URL` and `DATABASE_URL_POOLED` placeholder rows ADR-0048 records, never a name the app reads. +- With a preview-wide row made with `--role preview`, a fresh preview branch deploys with the value, and the service receives it. +- Setting `X`'s branch row and deploying again replaces exactly the deployments that read `X`. Changing the class row while a branch row exists replaces none. +- A row with `isManagedBySystem` under a name the app reads fails the deploy naming it. +- `prisma dev` with a required secret unset stops before any service starts, naming it; with an optional one unset, the service's document leaves the key out, as the deploy's would; a service receives only the variables its own markers and rows name. +- A secret planted in `.env` and in a value: the deploy output, the deploy report, the stack file and every error hold no part of it. + +## 4. Scope + +### In + +- Core: `checkGraph` on `ExtensionDescriptor`, called from `runPipeline`; the preflight contract's findings in place of counts; the new CLI event and both renderers; the transport carrying the findings to the deploy report. +- The Prisma Cloud target: the name checker, the graph check, the serializer, boot, preflight, the compute descriptor's `input` detail and triggers, `envParam()`'s deprecation tag. +- The shared package `@internal/input-document` and the local target's move onto it. +- The local target and `prisma dev`: the per-key param rule, empty means unset. +- Tests: every test the grounding report names as asserting the old behaviour changes with it; new tests for each rule in section 3, including a planted-secret test for deploy and boot errors. +- Records, guides, skill and release note as 2.9 and 2.10 list them. `design.md` section 14's ADR-0064 title, and `plan.md`'s named-secrets sentence, corrected. + +### Out + +- The node, `variable()` and `variable.secret()`. Slice 3. +- `schema`, decoding beyond text, the validator. Slice 4. +- Removing `envSecret()` or `envParam()`; deprecating `envSecret()`. Slice 3 deprecates it once the node exists. +- Carried secrets and `secret(inner)`. Module-input S18. +- Any platform change, and anything about a process that exits at boot on the platform. + +## 5. Done when + +CI is green, `/drive-code-review` has run on the slice with its findings fixed, and manual QA shows, against Prisma Cloud with a repository example: an `envSecret()` a service requires, unset, refused by preflight before any service or database is created, with the rows listed before and after unchanged; one nothing requires, unset, with the deploy succeeding and the two lists printed; set, then changed on its branch row, with only its readers redeployed; a fresh preview branch deploying with a `--role preview` row; a planted secret found in no output; and under `prisma dev`, a required secret unset stopping dev before any service starts, and a service receiving only its own variables. + +## 6. Open questions + +None. The choices marked **(decided here)** are engineering decisions inside the design; the design itself is unchanged except the ADR-0064 title. + +## 7. References + +- Design: [../../design.md](../../design.md) sections 3, 5, 6, 8 to 14. Grounding: `wip/pv-review/slice2-grounding.md`. +- `packages/0-framework/3-tooling/cli/src/pipeline.ts`, `operations/execute-deploy-destroy.ts`, `operations/execute-dev.ts`, `family/commands/deploy.ts`, `render-deployment.ts`, `app-configuration-file.ts`. +- `packages/0-framework/1-core/core/src/control/app-config.ts`, `preflight-transport.ts`, `resolve-values.ts` (the plain-or-secret check), `graph.ts` (`secretsKeys`). +- `packages/1-prisma-cloud/1-extensions/target/src/`: `secret.ts`, `param.ts`, `serializer.ts`, `compute.ts`, `preflight.ts`, `preflight-names.ts`, `local-target/preflight.ts`, `descriptors/compute.ts`, `control/extension.ts`, `control/preflight-payload.ts`, `__tests__/invariants.test.ts`. +- `packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts`; `packages/1-prisma-cloud/0-lowering/s3-protocol` as the model for the shared package; `architecture.config.json`. +- ADR-0029, 0032, 0042, 0058, 0062; `docs/design/03-domain-model/glossary.md` lines 249 to 269; `docs/design/00-purpose/goals.md`; `docs/guides/deploying.md`, `building-an-app.md`; `docs/design/10-domains/deploy-cli.md`; `skills/prisma-composer-core-concepts/SKILL.md`. diff --git a/.drive/projects/platform-variables/spec.md b/.drive/projects/platform-variables/spec.md index 4ad1fcf54..21d4a072e 100644 --- a/.drive/projects/platform-variables/spec.md +++ b/.drive/projects/platform-variables/spec.md @@ -1,6 +1,6 @@ # Platform variables: project spec -A service reads some of its configuration from values the operator keeps on the platform. This project lets an app name those values in its graph, type them, and receive them at boot, while the deploy never carries, stores or needs them. +A service reads some of its configuration from values the operator keeps on the platform. This project lets an app name those values in its graph, type them, and receive them at boot, while the deploy does not copy them from the developer's local environment, store them or need them. Design: [design.md](design.md), which is authoritative and holds the detail behind every statement here. Plan: [plan.md](plan.md), which says which slice builds each part. @@ -57,7 +57,7 @@ Each decision is stated once. The design.md section holds the detail. "Satisfies **D10. The input document carries a `$platform` marker, and boot follows four steps.** `{ "$platform": { "name", "secret"?, "schema"? } }` replaces today's `$secret` marker, for `envSecret()` too. At boot: a variable with no value leaves its key out and the service's schema decides whether that is allowed; a variable with text is decoded and validated; a secret is boxed; the service's schema validates the whole. An error names the variable and what was expected, never the text, a part of it, or a parser's or validator's message. An empty value cannot exist on the platform, so a variable is present with text or absent; `prisma dev` treats an empty value as unset, so both agree. Design section 8. Satisfies R3, R4, R7. -**D11. The deploy writes no variable, and refuses only a required variable with no row.** Preflight no longer seeds a missing variable from the app secrets or the app configuration. For each variable it reads whether a row exists at any scope the platform offers (project class and branch today; one more list call when the platform adds a service scope) and when it last changed; it never reads a value. It decides "required" by resolving each service's binding with every unset variable left out and running the service's schema, the method ADR-0058 gives the dev preflight. A required variable with no row fails the deploy before anything is created, naming the variable, the reader and the command that sets it. An optional one is reported, after preflight and again in the deploy report, with `prisma project env add … --role production` or `--role preview`. A changed value redeploys exactly its readers, through the deployment triggers. This check is the only protection Composer gives against a missing value: the platform reports a process that exits at boot as running and promotes it, so the guide says so and names `prisma service logs ` as where the boot error is. Design sections 3 and 9. Satisfies R2, R6. +**D11. The deploy does not copy local values to platform variables, and refuses only a required variable with no row.** Preflight no longer seeds a missing variable from the app secrets or the app configuration. An implicit copy can push a local value into a remote environment by accident: `.env` is read for every environment, and copying on each deploy also writes one override row per preview branch. For now the operator sets each value with `prisma project env add`. This is not a rule that the deploy may never set a variable: an explicit way to set variables through `deploy`, where the operator says which values go to which environment, is expected later (deferred.md). For each variable it reads whether a row exists at any scope the platform offers (project class and branch today; one more list call when the platform adds a service scope) and when it last changed; it never reads a value. It decides "required" by resolving each service's binding with every unset variable left out and running the service's schema, the method ADR-0058 gives the dev preflight. A required variable with no row fails the deploy before anything is created, naming the variable, the reader and the command that sets it. An optional one is reported, after preflight and again in the deploy report, with `prisma project env add … --role production` or `--role preview`. A changed value redeploys exactly its readers, through the deployment triggers. This check is the only protection Composer gives against a missing value: the platform reports a process that exits at boot as running and promotes it, so the guide says so and names `prisma service logs ` as where the boot error is. Design sections 3 and 9. Satisfies R2, R6. **D12. `prisma dev` plays the platform's part.** It fills a plain platform variable from the app configuration and a platform secret from the app secrets, under the variable's name; gives each local service only the variables its own rows and markers point at; decodes and validates before any service starts, with the shared implementation; applies the ADR-0058 missing-value rule; and restarts a service whose value changes. Design section 10. Satisfies R5. @@ -118,7 +118,7 @@ R1 to R7 are the design's requirements, numbered as in design section 2. R8 is t Each slice merges to `main` on its own. This is what is true after each. - **After slice 1.** `appRoot()` exists and every root in this repository and its guidance uses it. An entry built with `module()` still deploys, exactly as before. A root with `deps` or `expose`, or a root passed to `provision()`, fails to compile. Nothing about platform variables has changed: `envSecret()` and `envParam()` work as before, including preflight seeding a missing variable. -- **After slice 2.** The deploy fills nothing. Every operator variable that exists on `main`, which means every `envSecret()` and `envParam()`, follows the new rules: upper-case names, reserved names, one meaning per name, the `$platform` marker, the four boot steps, required-or-reported in preflight, existence at every scope the platform offers, triggers on `updatedAt`, and the `prisma dev` rules. `envParam()` in a binding is deprecated. The node does not exist yet, so `envSecret()` is the only way to name a platform secret and there is no plain platform variable. Every value is text. +- **After slice 2.** The deploy does not copy local values to platform variables. Every operator variable that exists on `main`, which means every `envSecret()` and `envParam()`, follows the new rules: upper-case names, reserved names, one meaning per name, the `$platform` marker, the four boot steps, required-or-reported in preflight, existence at every scope the platform offers, triggers on `updatedAt`, and the `prisma dev` rules. `envParam()` in a binding is deprecated. The node does not exist yet, so `envSecret()` is the only way to name a platform secret and there is no plain platform variable. Every value is text. - **After slice 3.** `variable()` and `variable.secret()` exist, only an `appRoot()` body can provision them, `provision()` returns the value, the graph records the reads relation, and the examples and guides use the nodes. `envSecret()` is deprecated and fails the graph check below the root. `schema` does not exist; every value is text. - **After slice 4.** `schema` exists on `variable()`, with the decoding rule and the validator. - **At every point before module-input S18 ships,** a platform secret binds to a `secretString()` field. After it ships, to `secret(inner)`. This project does not depend on S18. @@ -160,7 +160,7 @@ The project-level ones. Design section 15 has the full list with reasons. - **Root-only enforced by a Load check alone,** with `module()` as the root. Rejected by Will: the rule is structural, so the type system carries it; `appRoot()` makes the misuse fail to compile. - **A source with no node,** as `envSecret()` is, or a new node kind. Rejected: a source leaves nothing for the graph to list; a new kind changes core's node union and the platform's topology kinds for the same result. - **`provision()` returning `{ id, value }`.** Rejected in favour of the value itself with `id` on it, matching `generatedParam()` and `serviceOrigin()`. -- **Keep seeding on deploy, or add a seeding command.** Rejected: seeding on deploy is how a local value reaches production; a seeding command was shelved by Will on 2026-11-05. +- **Keep seeding on deploy.** Rejected: an implicit copy is how a local value reaches a remote environment by accident. An explicit way to set variables through `deploy` is expected later (deferred.md). - **Decode by trying JSON first, or a fixed vocabulary of types.** Rejected by Will: Composer must not guess, and a standard carries the type in a form any library can produce. - **Wait for the platform to record who reads what, or to add a service scope.** Not needed: every part ships against today's API, and either change would alter preflight's list calls and nothing an author writes. diff --git a/.github/actions/deploy-verify-destroy/action.yml b/.github/actions/deploy-verify-destroy/action.yml index 1e80e05f2..16e0e2c7c 100644 --- a/.github/actions/deploy-verify-destroy/action.yml +++ b/.github/actions/deploy-verify-destroy/action.yml @@ -29,6 +29,13 @@ inputs: description: Space-separated project-name prefixes for ci-cleanup.ts required: false default: storefront-auth orm-demo hello + platform-variables: + description: > + Space-separated names of platform variables the app needs. When set, the + action generates a random value for each, checks that the first deploy + stops at preflight naming each one, sets them, and deploys again. + required: false + default: '' runs: using: composite @@ -46,6 +53,24 @@ runs: working-directory: ${{ inputs.working-directory }} run: pnpm turbo run build --filter '${{ inputs.build-filter }}' + - name: Generate platform variable values + if: inputs.platform-variables != '' + shell: bash + env: + PLATFORM_VARIABLES: ${{ inputs.platform-variables }} + run: bun "$GITHUB_WORKSPACE/scripts/e2e-generate-platform-variables.ts" $PLATFORM_VARIABLES + + - name: First deploy stops at preflight, then set platform variables + id: first-deploy + if: inputs.platform-variables != '' + shell: bash + working-directory: ${{ inputs.working-directory }} + env: + STACK_NAME: ${{ inputs.stack-name }} + ENVIRONMENT: ${{ inputs.environment }} + PLATFORM_VARIABLES: ${{ inputs.platform-variables }} + run: bun "$GITHUB_WORKSPACE/scripts/e2e-first-deploy.ts" $PLATFORM_VARIABLES + - name: Deploy id: deploy # Must run under bun: Load imports service modules that import `bun` @@ -81,9 +106,7 @@ runs: run: bun "$GITHUB_WORKSPACE/scripts/e2e-verify-deploy-list.ts" - name: Destroy - # Check the Deploy step's outcome, not `.alchemy/` presence — deploy - # state is hosted, not local to the runner. - if: always() && steps.deploy.outcome != 'skipped' + if: always() && (steps.deploy.outcome != 'skipped' || steps.first-deploy.outcome != 'skipped') shell: bash working-directory: ${{ inputs.working-directory }} env: diff --git a/.github/workflows/e2e-deploy.yml b/.github/workflows/e2e-deploy.yml index 24fc5bf1c..53b035735 100644 --- a/.github/workflows/e2e-deploy.yml +++ b/.github/workflows/e2e-deploy.yml @@ -27,19 +27,12 @@ jobs: deploy: name: Deploy, verify, destroy runs-on: ubuntu-latest - timeout-minutes: 5 + timeout-minutes: 7 # Secrets aren't available on forked-PR runs; skip rather than fail loudly. if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository env: PRISMA_SERVICE_TOKEN: ${{ secrets.PRISMA_SERVICE_TOKEN }} PRISMA_WORKSPACE_ID: ${{ vars.PRISMA_WORKSPACE_ID }} - # The root binds the auth module's signingKey input to AUTH_SIGNING_SECRET (ADR-0029). - # This non-sensitive test marker in the runner env is what deploy preflight - # fill-missing provisions onto the ephemeral per-run stack — the single-step - # CI path (runner env -> direct API POST -> platform). It must match the - # auth service's EXPECTED_SIGNING_SECRET (modules/auth/src/server.ts); the - # value never enters deploy state, only its pointer name does. - AUTH_SIGNING_SECRET: sk_test_ci_storefront_auth steps: - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 with: @@ -49,7 +42,8 @@ jobs: working-directory: examples/storefront-auth build-filter: '@prisma/example-storefront-auth...' stack-name: storefront-auth-ci-${{ github.run_id }} - verify-command: bash scripts/e2e-verify.sh + verify-command: bun scripts/e2e-verify.ts + platform-variables: AUTH_SIGNING_SECRET deploy-orm-demo: name: Deploy, verify, destroy (orm-demo) diff --git a/architecture.config.json b/architecture.config.json index c6bae9ce5..7b3814a5d 100644 --- a/architecture.config.json +++ b/architecture.config.json @@ -138,6 +138,12 @@ "layer": "lowering", "plane": "shared" }, + { + "glob": "packages/1-prisma-cloud/0-lowering/input-document/src/**", + "domain": "prisma-cloud", + "layer": "lowering", + "plane": "shared" + }, { "glob": "packages/1-prisma-cloud/0-lowering/dev-emulators/src/**", "domain": "prisma-cloud", @@ -396,6 +402,36 @@ "layer": "extensions", "plane": "shared" }, + { + "glob": "packages/1-prisma-cloud/1-extensions/target/src/map-with-concurrency.ts", + "domain": "prisma-cloud", + "layer": "extensions", + "plane": "control" + }, + { + "glob": "packages/1-prisma-cloud/1-extensions/target/src/required-variables.ts", + "domain": "prisma-cloud", + "layer": "extensions", + "plane": "control" + }, + { + "glob": "packages/1-prisma-cloud/1-extensions/target/src/missing-variables-text.ts", + "domain": "prisma-cloud", + "layer": "extensions", + "plane": "control" + }, + { + "glob": "packages/1-prisma-cloud/1-extensions/target/src/graph-check.ts", + "domain": "prisma-cloud", + "layer": "extensions", + "plane": "control" + }, + { + "glob": "packages/1-prisma-cloud/1-extensions/target/src/variable-name.ts", + "domain": "prisma-cloud", + "layer": "extensions", + "plane": "shared" + }, { "glob": "packages/1-prisma-cloud/1-extensions/target/src/preflight-names.ts", "domain": "prisma-cloud", diff --git a/docs/design/00-purpose/goals.md b/docs/design/00-purpose/goals.md index e7211f27d..55ad0337e 100644 --- a/docs/design/00-purpose/goals.md +++ b/docs/design/00-purpose/goals.md @@ -15,9 +15,7 @@ from the purpose, not the other way around. ships the Prisma Cloud pack, and the core stays target-agnostic. - **A queryable topology.** The inferred topology is a first-class artifact you and your agents can interrogate from the CLI. -- **Recreate the whole topology in a fresh environment.** Every element has a - managed lifecycle, so standing up a new environment provisions the entire module - from nothing — no click-ops, no manually-created prerequisites. +- **Recreate the whole topology in a fresh environment.** Every element has a managed lifecycle, so standing up a new environment provisions the entire module from nothing, with no click-ops. The one prerequisite is the platform variables the app reads, such as API keys: for now the operator sets them on the platform first, because the deploy does not copy local values to the platform. Before any service or database is created, the deploy names every platform variable a service needs that has no value, with the command that sets it. - **Reproduce every element in the local dev emulator.** The same topology runs locally: each Resource ships a local stand-in beside its real provider, so `prisma dev` emulates the deployed module without touching the cloud. diff --git a/docs/design/03-domain-model/glossary.md b/docs/design/03-domain-model/glossary.md index e599ca08e..d10427909 100644 --- a/docs/design/03-domain-model/glossary.md +++ b/docs/design/03-domain-model/glossary.md @@ -246,9 +246,13 @@ schema whose leaves are wrapped. For another Standard Schema library, `@prisma/composer` exports `Pointer`, `isPlainPointer()` and `isSecretPointer()`, with which an author writes the same two helpers. A service reads a wrapped field as `Out`: `service.input()` returns the schema's output with every pointer removed, because at boot each pointer has been replaced by the text of the variable it names. +### Platform variable + +A variable the operator sets on the platform and a service reads at boot. It is named with `envSecret()`, or with `envParam()` in a service's per-key params. A deploy does not copy its value from the developer's local environment and does not read it: the platform hands it to the running service. For now the operator sets it with `prisma project env add`. Under `prisma dev`, the local target plays the platform's part: it reads the value from the app secrets or the app configuration and hands it to the local service. A variable has a class row (production or preview) and, on a preview environment, may have a branch row for one branch; the branch row wins ([ADR-0064](../90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). In code: `PlatformVariableReader`, `UnsetPlatformVariable`, `PlatformVariableCheck`. + ### The app configuration -The merged values the `config` adapters return for the environment, a later adapter winning by top-level key. "App" is what Composer calls the application, the outermost module, whose input it fills. It is the root module's input, checked by the root's schema, and what an `envParam('NAME')` reads `NAME` from, wherever it sits; the name a source reads is its **configuration key** (`paramSource()`'s `configurationKey`). With no `config` list it is the process environment, secrets included, so only the root fields its body passes on by name reach the graph. Only a command's first process reads it; the deploy's and `prisma dev`'s second process reads what the first process's Load read from the **app configuration file**. In code: `AppConfiguration`, `LoadOptions.configuration`. +The merged values the `config` adapters return for the environment, a later adapter winning by top-level key. "App" is what Composer calls the application, the outermost module, whose input it fills. It is the root module's input, checked by the root's schema, and what an `envParam('NAME')` reads `NAME` from, wherever it sits; the name a source reads is its **configuration key** (`paramSource()`'s `configurationKey`). Nothing fills a [platform variable](#platform-variable) from it: the operator sets a platform variable on the platform. With no `config` list it is the process environment, secrets included, so only the root fields its body passes on by name reach the graph. Only a command's first process reads it; the deploy's and `prisma dev`'s second process reads what the first process's Load read from the **app configuration file**. In code: `AppConfiguration`, `LoadOptions.configuration`. ### Configuration key @@ -256,19 +260,19 @@ The name in the app configuration that a param source reads: `envParam('NAME')` ### The app secrets -The merged values the `secrets` adapters return, text only. The name a secret source reads is its **secrets key** (`secretSource()`'s `secretsKey`; `envSecret('NAME')` passes `NAME`). Deploy preflight fills a missing `envSecret()` platform variable from it. `prisma dev` hands its second process only the **named secrets**, the values under the secrets keys the graph's `envSecret()`s name, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS` (in code: `NamedSecrets`). The app secrets hold values Composer may carry; only the values under the secrets keys the graph names become known secrets. `prisma dev` compares the whole app secrets between rebuilds to notice a change, which reads them without making them known secrets. Composer never passes the app secrets to Load or a schema, never writes their values to a file, and redacts the values under the secrets keys wherever it prints. With no `secrets` list it is the process environment. In code: `AppSecrets`. +The merged values the `secrets` adapters return, text only. The name a secret source reads is its **secrets key** (`secretSource()`'s `secretsKey`; `envSecret('NAME')` passes `NAME`). The deploy does not copy the app secrets to a [platform variable](#platform-variable): the operator sets a platform variable on the platform, and the deploy reads the app secrets only to know which values to redact. `prisma dev` hands its second process only the **named secrets**, the values under the secrets keys of the graph's secret sources, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS` (in code: `NamedSecrets`). The app secrets hold values Composer may carry; only the values under the secrets keys the graph names become known secrets. `prisma dev` compares the whole app secrets between rebuilds to notice a change, which reads them without making them known secrets. Composer never passes the app secrets to Load or a schema, never writes their values to a file, and redacts the values under the secrets keys wherever it prints. With no `secrets` list it is the process environment. In code: `AppSecrets`. ### Secrets key -The name in the app secrets that a secret source reads: `envSecret('NAME')` reads `NAME`. See [the app secrets](#the-app-secrets). In code: `secretSource()`'s `secretsKey`, and `secretsKeys(graph)`, every secrets key a graph reads. +The name in the app secrets that a secret source reads: `envSecret('NAME')` reads `NAME`, which is also the name of the [platform variable](#platform-variable) the service reads at boot. Nothing writes the value under it to the platform. See [the app secrets](#the-app-secrets). In code: `secretSource()`'s `secretsKey`, and `secretsKeys(graph)`, every secrets key of the graph's secret sources, including a secret in a module's input that no service reads. ### Named secrets -The app secrets under the secrets keys the graph reads, each with text. Only `prisma dev` hands them to its second process, in one environment variable. See [the app secrets](#the-app-secrets). In code: `NamedSecrets`, `PRISMA_COMPOSER_NAMED_SECRETS`. +The app secrets under the secrets keys of the graph's secret sources, each with text. Only `prisma dev` hands them to its second process, in one environment variable, and the local target gives each service the ones its own markers name. Nothing fills a [platform variable](#platform-variable) from them. See [the app secrets](#the-app-secrets). In code: `NamedSecrets`, `PRISMA_COMPOSER_NAMED_SECRETS`. ### Known secrets -The values one command knows are secrets: the value under each secrets key the graph's `envSecret()`s name, in the app secrets or the app configuration. Nothing else is a known secret: a plain value in the same `.env` file, or the shell's `SHLVL=1` under `secrets: [dotenv(), environmentVariables()]`, is not. The CLI refuses to write the app configuration file when a value in it contains a known secret: one of eight characters or more anywhere in the value, a shorter one only as the whole value. The CLI redacts the known secrets wherever it prints, together with **credential-looking values**: text of eight characters or more under a variable name that looks like a credential's, which might not be secrets ([ADR-0062](../90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)). In code: `KnownSecrets`, and `SecretRedaction`, which holds both kinds. +The values one command knows are secrets: the value under each secrets key of the graph's secret sources, in the app secrets or the app configuration. Composer reads them on a deploy only to redact them; nothing fills a [platform variable](#platform-variable) from them. Nothing else is a known secret: a plain value in the same `.env` file, or the shell's `SHLVL=1` under `secrets: [dotenv(), environmentVariables()]`, is not. The CLI refuses to write the app configuration file when a value in it contains a known secret: one of eight characters or more anywhere in the value, a shorter one only as the whole value. The CLI redacts the known secrets wherever it prints, together with **credential-looking values**: text of eight characters or more under a variable name that looks like a credential's, which might not be secrets ([ADR-0062](../90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)). In code: `KnownSecrets`, and `SecretRedaction`, which holds both kinds. ### The app configuration file diff --git a/docs/design/10-domains/config-params.md b/docs/design/10-domains/config-params.md index dfa8c0c1f..15d455477 100644 --- a/docs/design/10-domains/config-params.md +++ b/docs/design/10-domains/config-params.md @@ -92,18 +92,7 @@ Resolution order per param: binding, else `default`, else absent (only legal for an `optional` param) — a required param nothing binds fails the deploy loudly, naming the param and the service. -An **env-sourced** param mirrors the secrets rail (§ Secrets): the stored row is -a pointer to the platform variable's NAME (discriminated from a JSON-encoded -literal by the `@composer-param-pointer:` value prefix, which no JSON output -can start with), deploy preflight verifies the name exists per environment (filling -it from the app configuration when absent, failing early otherwise), and boot does -the double-lookup — pointer, then platform variable. The differences from a -secret are deliberate: the value reads back through `config()`, **unredacted**; -the platform variable's **raw string** is handed to the param's own schema (no -decode — a `number()` param bound to `envParam` fails at boot); and an empty -string is a value, valid iff the schema accepts it. A module passes a binding -down to a child as a field of its own `input`. Value changes need a redeploy, -same as secret rotation. +An **env-sourced** param is stored like a secret (§ Secrets). Its stored row holds the name of the platform variable, not a value. A `@composer-param-pointer:` prefix marks the row as a name; no JSON-encoded literal can start with that prefix. The operator sets the variable on the platform. The deploy checks whether the variable has a value in the environment, and does not set it. At boot, the service reads the row to find the variable's name, then reads that variable. A variable with no value gives the param its default, or leaves it out when the param is optional. The deploy check and boot fail only for a required param ([ADR-0064](../90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). Three things differ from a secret on purpose. The value reads back **unredacted**. The param's own schema receives the variable's **raw text**, with no decoding, so a `number()` param bound to `envParam` fails at boot. An empty string means unset, because the platform cannot store one. A module passes a binding down to a child as a field of its own `input`. A changed value needs a redeploy, as a rotated secret does. **One concept, a param, bound to a source** (the glossary's *params and their sources* is the canonical statement). Four sources, by where the value comes @@ -291,23 +280,14 @@ secrets). The round trip carries only the name, never the value: -1. **Bind.** The root binds each secret field to a platform name, and each - module on the way passes it on as a field of its input. Load records - each service's input binding, and preflight collects every `envSecret()` it - names. -2. **Preflight.** Before Alchemy runs, the deploy verifies every bound name - exists on the platform for the target environment's class/branch, filling a name - from the app secrets when the platform lacks it (a direct write-only POST, - never an Alchemy resource, never overwriting an existing value). A name - missing from both fails the deploy, listing exactly what's absent. -3. **Pointer row.** The target writes a pointer per secret field — the generated key - (`COMPOSER_`-prefixed, like every generated key) maps to the bound name, not a - value: +1. **Bind.** The root binds each secret field to a platform name, and each module on the way passes it on as a field of its input. Load records each service's input binding, and preflight collects every `envSecret()` it names. +2. **Set.** The operator sets the variable on the platform with `prisma project env add`, for production, for every preview environment, or for one. The deploy does not copy a value from the app secrets to the platform. +3. **Preflight.** Before Alchemy runs, the deploy checks whether each name has a row the environment sees, and writes none. A missing name that a service's schema requires fails the deploy, with its readers and the command that sets it; a missing name nothing requires is listed and the deploy goes on. +4. **Marker.** The target writes the service's input document with a marker in place of each secret field, naming the variable, not its value: + ```json + { "signingKey": { "$platform": { "name": "AUTH_SIGNING_KEY", "secret": true } } } ``` - COMPOSER_SHOP_AUTH_SIGNINGKEY = "AUTH_SIGNING_KEY" # service `auth` in the app `shop` - ``` -4. **Boot double-lookup.** Boot reads the pointer for the name, then reads - `process.env[name]` for the platform value, and wraps it in a `SecretBox`. +5. **Boot.** Boot reads `process.env[name]` for each marker and wraps the text in a `SecretBox`. An unset or empty variable leaves the field out, and the service's schema decides. The `COMPOSER_` prefix reserves the framework's generated keys into their own namespace, so a generated key never collides with — and silently overwrites — a diff --git a/docs/design/10-domains/deploy-cli.md b/docs/design/10-domains/deploy-cli.md index b8d089859..e0106a672 100644 --- a/docs/design/10-domains/deploy-cli.md +++ b/docs/design/10-domains/deploy-cli.md @@ -129,11 +129,9 @@ user can rerun it by hand (ADR-0007). digits, `-` and `_`; Load refuses any other. An app that uses `serviceOrigin()` can't be deployed under `--name`: the addresses in its code start with its own name, which then names no node. -4. **Read the app secrets and the app configuration.** For `deploy` the CLI first calls the `secrets` adapters, then for every command the `config` adapters, in list order, each awaited, and merges what each list returns by top-level key, a later adapter winning. Both lists default to `[environmentVariables()]`. The app configuration is Load's input: the root's input and what `envParam()` reads. The app secrets are held in memory for preflight. Once Load has run, the values under the names an `envSecret()` in the graph reads, in either list, become the known secrets the CLI redacts; nothing else in the app secrets is read. An adapter that throws or returns something other than an object fails with `CONFIG.ADAPTER_FAILED` or `CONFIG.ADAPTER_INVALID`. No other process runs an adapter. +4. **Read the app secrets and the app configuration.** For `deploy` the CLI first calls the `secrets` adapters, then for every command the `config` adapters, in list order, each awaited, and merges what each list returns by top-level key, a later adapter winning. Both lists default to `[environmentVariables()]`. The app configuration is Load's input: the root's input and what `envParam()` reads. The app secrets are held in memory. A deploy does not write a value from them to the platform. Once Load has run, the values under the secrets keys of the graph's secret sources, in either list, become the known secrets the CLI redacts, and nothing else in the app secrets is read. An adapter that throws or returns something other than an object fails with `CONFIG.ADAPTER_FAILED` or `CONFIG.ADAPTER_INVALID`. No other process runs an adapter. 5. **Load.** Core's `Load` walks the graph. The root's schema checks the app configuration as the root's input before anything is created. A service with an unwired dependency slot (one an enclosing Module normally wires to a provisioned producer) fails here, with an error naming the input and pointing at the composing Module. -6. **Validate coverage.** Every node's and build descriptor's - `(extension, type)` must have a registry entry in the config; a gap errors - naming the extension to add to the config. +6. **Check the graph, then validate coverage.** Every extension's optional `checkGraph(graph)` runs first, in `deploy`, `destroy` and `prisma dev` (at start and at every rebuild); a check refuses a graph by throwing core's `GraphCheckError`, which fails the command with `COMPOSE.GRAPH_INVALID` and its message; any other throw is a bug. The Prisma Cloud extension refuses a name that both an `envParam()` and a field of the root's input use ([ADR-0064](../90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). Then every node's and build descriptor's `(extension, type)` must have a registry entry in the config; a gap errors naming the extension to add to the config. 7. **Assemble each service.** Look up the service's build descriptor in the registries — `extensions[build.extension].nodes[build.type].assemble` — and run it. The assembler resolves its `entry` (and any other path field) @@ -174,15 +172,8 @@ user can rerun it by hand (ADR-0007). emulator instance is touched. On `deploy`, an older extension's `ensure` may already have created the app's Project by then; the check stops everything after that. -9. **Lower and drive.** Write the pipeline's results as a runnable stack - module at `.prisma-composer/alchemy.run.ts` and drive the `alchemy` CLI against - it (ADR-0007), started as § Runtime describes, carrying every resolved container across to the child as one - environment variable per extension — content the CLI writes but never - reads (ADR-0038). Both `deploy` and `destroy` set it, since `alchemy - destroy` re-imports and re-evaluates the same stack, so its target - reconstruction needs the same containers. The generated file and Alchemy's - state live in the process's working directory — tool state lives where you - run the tool, like any other CLI (ADR-0004). Beside it the CLI writes the app configuration file, `.prisma-composer/app-configuration--.json`, mode 0600, and deletes it when the run ends. It holds what Load read from the app configuration and the first process's node addresses; it is not `prisma.config.ts`. The CLI passes its path in `PRISMA_COMPOSER_APP_CONFIGURATION_FILE`. The second process reads it, runs no adapter, and refuses a graph whose node addresses differ from the first process's. It never receives the app secrets. The CLI refuses to write the file when any value in it contains a known secret (`CONFIG.SECRET_AS_PLAIN_VALUE`). It also deletes the file when its process exits. At the start of each `deploy` and `destroy` it deletes the app configuration files an earlier run left in `.prisma-composer/`, when the process id in the file name is not running and the file is older than ten minutes. The CLI engine gives it the terminal, so its output is not redacted; Composer redacts its failure cause. +9. **Preflight (`deploy` only).** Each extension's `preflight` checks its platform prerequisites before any stack file is written. The Prisma Cloud preflight checks that every platform variable a service needs has a value, and sets none. A variable a service requires that has no value fails the deploy with `DEPLOY.PREFLIGHT_FAILED`, naming the services that read it and the `prisma project env add` commands that set it. The CLI lists the other variables with no value after the step, under `Platform variables with no value:`, and again at the end of the deploy report. The platform reports a service that exits at boot as running and promotes it, so this step is the only check Composer makes for a missing value ([ADR-0064](../90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). +10. **Lower and drive.** Write the pipeline's results as a runnable stack module at `.prisma-composer/alchemy.run.ts` and drive the `alchemy` CLI against it (ADR-0007), started as § Runtime describes, carrying every resolved container across to the child as one environment variable per extension — content the CLI writes but never reads (ADR-0038). Both `deploy` and `destroy` set it, since `alchemy destroy` re-imports and re-evaluates the same stack, so its target reconstruction needs the same containers. The generated file and Alchemy's state live in the process's working directory — tool state lives where you run the tool, like any other CLI (ADR-0004). Beside it the CLI writes the app configuration file, `.prisma-composer/app-configuration--.json`, mode 0600, and deletes it when the run ends. It holds what Load read from the app configuration and the first process's node addresses; it is not `prisma.config.ts`. The CLI passes its path in `PRISMA_COMPOSER_APP_CONFIGURATION_FILE`. The second process reads it, runs no adapter, and refuses a graph whose node addresses differ from the first process's. It never receives the app secrets. The CLI refuses to write the file when any value in it contains a known secret (`CONFIG.SECRET_AS_PLAIN_VALUE`). It also deletes the file when its process exits. At the start of each `deploy` and `destroy` it deletes the app configuration files an earlier run left in `.prisma-composer/`, when the process id in the file name is not running and the file is older than ten minutes. The CLI engine gives it the terminal, so its output is not redacted; Composer redacts its failure cause. The pass that assembles a service is the same pass that lowers it, so the correlation between services and their built bundles never exists as @@ -293,6 +284,9 @@ them: | Node `(extension, type)` not covered | the extension to add to the `composer` section | | Missing extension env | the exact variable(s) the extension factory needed | | Built output missing | the expected path, and "run your build" | +| An extension's graph check refuses the graph (`COMPOSE.GRAPH_INVALID`) | the extension's message: for Prisma Cloud, a name used by both `envParam()` and a root input field, naming both uses | +| A platform variable a service needs has no value (`DEPLOY.PREFLIGHT_FAILED`) | each variable, the service and field or param that reads it, and the `prisma project env add` commands: `--role production` for production, otherwise `--role preview` then `--role preview --branch` | +| A platform variable the app reads is managed by Prisma Cloud (`DEPLOY.PREFLIGHT_FAILED`) | the variable, and to rename it in the app | | Root's input refused by its schema | the schema's messages, and that a root's input is the app configuration for the environment | | A config adapter throws or returns something other than an object (`CONFIG.ADAPTER_FAILED`, `CONFIG.ADAPTER_INVALID`) | which list and position, and the fix | | The app configuration file would hold a known secret (`CONFIG.SECRET_AS_PLAIN_VALUE`) | which root input field or configuration key holds a known secret, and the secrets key whose value matched, never the value; a secret of eight characters or more counts anywhere in a value, a shorter one only as the whole value. The fix: read the secret with `envSecret()`, or give the plain value text that does not contain it | @@ -355,6 +349,4 @@ them: (§ Environments and containers). - [ADR-0059](../90-decisions/ADR-0059-a-deployment-environment-is-called-an-environment.md) — the term is environment. -- [ADR-0038](../90-decisions/ADR-0038-containers-are-an-extension-descriptor.md) - — the container descriptor and the opaque parent→child transport - (pipeline steps 8-9, § Environments and containers). +- [ADR-0038](../90-decisions/ADR-0038-containers-are-an-extension-descriptor.md) — the container descriptor and the opaque parent→child transport (pipeline steps 8 and 10, § Environments and containers). diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index b52adce0d..ba5906fc8 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -543,14 +543,20 @@ The same table the port's hand-rolled script implemented, now standard: | --- | --- | | dependency connection values (URLs) | the local provider's resolved value, through the normal lowering | | service params | bound literals / defaults, identical to deploy | -| `envParam` sources (ADR-0032) | the app configuration, same names; missing → a hard error listing the names (params feed boot-time schema validation — junk there is a confusing crash, not a legible degraded mode) | -| secrets (ADR-0029, ADR-0058) | the app secrets: by default the shell, or `dotenv()` in the `secrets` list. Unset or empty counts as missing. A missing secret the service's schema requires is a hard error naming the variable, the service and both ways to set it; a missing optional secret is left out of the service's input. No value is reused from an earlier run. | +| `envParam` sources (ADR-0032) | the app configuration, same names. An unset or empty per-key param takes its default or is left out when optional; a required one is a hard error listing the names. A value that is not text is a hard error naming the variable, the service and the kind of value | +| secrets (ADR-0029, ADR-0058) | the app secrets: by default the shell, or `dotenv()` in the `secrets` list. Unset or empty counts as missing. A missing secret the service's schema requires, or that an array position reads, is a hard error naming the variable, the service and both ways to set it; a missing optional secret is left out of the service's input. No value is reused from an earlier run. | | minted needs (ADR-0030/0031: service keys, streams keys) | minted locally by the unchanged provisioners | | origin (ADR-0039) | `http://localhost:` via the unchanged origin channel | -Whether a missing secret is required is decided by validation, not by reading the schema (ADR-0042): the dev preflight validates each service's input with every missing secret left out, and then with each one left out alone, and names as required only the secrets whose absence alone fails. The preflight sends the alchemy process, on the preflight transport, the names of the absent secrets and an HMAC-SHA256 fingerprint of the value of each variable that a service's input or params point at, keyed by the environment's own `.prisma-composer/dev//fingerprint.key`. The serializer leaves out the absent secrets, and the fingerprint fills each deployment's `:updatedAt` trigger member, so a value changed between two runs redeploys exactly the services that read it. The dev second process runs no adapter. It reads what Load read from the app configuration, and the first process's node addresses, from the app configuration file `.prisma-composer/dev//app-configuration-.json`, which `prisma dev` writes before each converge and deletes as soon as that converge ends, and receives the named secrets, the app secrets the graph's `envSecret()`s name, in `PRISMA_COMPOSER_NAMED_SECRETS`. The local `Deployment` provider gives each service only the variables its own input and params point at. +Whether a missing secret is required is decided by validation, not by reading the schema (ADR-0042). The dev preflight validates each service's input with every missing secret left out, then with each one left out alone. It names as required only the secrets whose absence alone fails. A secret an array position reads is required without a check, because an array position cannot be left out. The error opens with `1 platform variable a service needs has no value:` (or the count of them). -To load `.env` files, list `dotenv()` in `config` and `secrets`, with `environmentVariables()` after it to keep the shell. It reads `.env`, `.env.local`, `.env.` and `.env..local` beside `prisma.config.ts`, through `dotenv-flow`, and skips `.env.local` for the environment `test`. Values in `.env.dev` are read only by the environment `dev`; values in `.env` and `.env.local` are read by every environment, `prisma deploy` to `production` included, whose preflight uploads any secret the environment does not have yet. `import 'dotenv/config'` at the top of `prisma.config.ts` still works with the default lists; remove it when you list `dotenv()`. +The preflight passes two things to the second process in an environment variable: the names of the absent secrets, and an HMAC-SHA256 fingerprint of the value of each variable that a service's input or params read. The fingerprint is keyed by the environment's own `.prisma-composer/dev//fingerprint.key`. The serializer leaves out the absent secrets. Each fingerprint fills the `:updatedAt` trigger of the deployments that read the variable, so a value changed between two runs redeploys exactly the services that read it. + +The second process of `prisma dev` runs no adapter. It reads two things from the app configuration file, `.prisma-composer/dev//app-configuration-.json`: what Load read from the app configuration, and the first process's node addresses. `prisma dev` writes that file before each converge and deletes it as soon as the converge ends. The second process receives the named secrets in `PRISMA_COMPOSER_NAMED_SECRETS`: the app secrets under the secrets keys of the graph's secret sources. + +The local `Deployment` provider gives each service only the variables its own input and params read: the `$platform` markers in its input document, at any depth, and the variables its per-key params name. This isolation is a rule of the local target only. On Prisma Cloud every deployment receives every variable of its class. + +To load `.env` files, list `dotenv()` in `config` and `secrets`, with `environmentVariables()` after it to keep the shell. It reads `.env`, `.env.local`, `.env.` and `.env..local` beside `prisma.config.ts`, through `dotenv-flow`, and skips `.env.local` for the environment `test`. Values in `.env.dev` are read only by the environment `dev`; values in `.env` and `.env.local` are read by every environment, `prisma deploy` to `production` included. The deploy does not copy a secret to the platform, but a plain value there reaches the root's input and every `envParam()` of every environment. `import 'dotenv/config'` at the top of `prisma.config.ts` still works with the default lists; remove it when you list `dotenv()`. ## Scheduled work @@ -575,7 +581,7 @@ Deploy's rule holds: every failure names its fix. | a `@prisma/dev` server fails to start | the underlying error, sanitized — connection-URL credentials are masked before embedding | | port conflict on a persisted allocation | which service, which port, and how to free or re-allocate (`--fresh`) | | required secret unset or empty | hard error listing every missing variable, the service that needs it, the schema's message, and where an adapter in `secrets` reads it; an optional one is not an error | -| env-sourced param unbound | hard error listing the missing names, deploy-preflight style | +| a required env-sourced param unset or empty | hard error listing the missing names and the service and param that read each, in the same format as the deploy's error; one with a default or that is optional is not an error | | an adapter returns different values at a rebuild than at start | that the values changed and rebuilds are paused until dev restarts | | a service exits before it listens, or does not listen within the readiness timeout | the address, the port, the exit code if it exited, and the log path; `dev` stops the app's services before it returns the failure | | an emulator does not accept connections (retried briefly; an error response from the emulator is never retried) | `DEV.EMULATOR_UNREACHABLE`, with the emulator's URL, from `dev` and `log` alike | diff --git a/docs/design/90-decisions/ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md b/docs/design/90-decisions/ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md index 97beaed9d..f199f1bc3 100644 --- a/docs/design/90-decisions/ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md +++ b/docs/design/90-decisions/ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md @@ -26,14 +26,14 @@ Three pieces carry the model: **The framework carries the name, never the value.** At deploy, the service's input document gets a pointer, the name of the platform variable, not its contents: ```json -{ "baseUrl": "https://app.example", "signingKey": { "$secret": "AUTH_SIGNING_KEY" } } +{ "baseUrl": "https://app.example", "signingKey": { "$platform": { "name": "AUTH_SIGNING_KEY", "secret": true } } } ``` -At boot the framework swaps each pointer for a redacting box over the named variable, which the platform injects into the running instance. +At boot the framework swaps each pointer for a redacting box over the named variable, which the platform injects into the running instance. A variable that is unset or empty leaves its field out, and the schema decides whether that is allowed ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). -**Preflight fills a missing variable from the app secrets.** Before anything is provisioned, deploy preflight checks that every name an `envSecret()` in the graph names exists on the platform for the environment. A missing name is filled from the app secrets, what the `composer` section's `secrets` adapters return ([ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)), with one write-only API call, never recorded as managed infrastructure. A name the app secrets lack fails the deploy early, listing what to set. Preflight fills only a variable that does not exist yet; to change one, change it on the platform. +**The operator sets the variable on the platform; preflight only checks it.** The deploy does not copy a value from the app secrets or the shell to the platform ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). Before Alchemy runs, deploy preflight checks whether each name an `envSecret()` in the graph names has a row the environment sees. It reads no value and writes no variable. A missing variable that a service's schema requires fails the deploy, naming the variable, its readers and the `prisma project env add` commands that set it. A missing one nothing requires is listed and the deploy goes on ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). To change a value, change it on the platform and deploy again. -**Composer redacts every secret it knows.** Everywhere it prints, including errors, events, adapter failures, a thrown defect's cause chain and the deploy engine's failure cause, Composer replaces with `[redacted]` the value under every name an `envSecret()` in the graph names, in the app configuration and in the app secrets. Nothing else in the app secrets is a known secret. A known value is redacted at any length. When two known secrets overlap, it hides every character either one covers. Nothing is redacted by the known secrets before Load has named them: an adapter that throws, and a root schema failure that echoes a field's value, are redacted only by the credential-looking rule. Whether the deploy engine's own output is redacted depends on who runs it ([ADR-0062, "Alchemy's output"](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md#keeping-secrets-out)). Composer also still redacts what it redacted before this decision: a value of eight characters or more under a name that looks like a credential, in the app configuration or the process environment, bearer and basic tokens, JWTs, a password in a URL and `token=` pairs. That is existing behaviour, kept until secrets get a home apart from plain values. +**Composer redacts every secret it knows.** Everywhere it prints, including errors, events, adapter failures, a thrown defect's cause chain and the deploy engine's failure cause, Composer replaces with `[redacted]` the value under each secrets key of the graph's secret sources (`envSecret('NAME')` passes `NAME`), in the app configuration and in the app secrets. Nothing else in the app secrets is a known secret. A known value is redacted at any length. When two known secrets overlap, it hides every character either one covers. Nothing is redacted by the known secrets before Load has named them: an adapter that throws, and a root schema failure that echoes a field's value, are redacted only by the credential-looking rule. Whether the deploy engine's own output is redacted depends on who runs it ([ADR-0062, "Alchemy's output"](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md#keeping-secrets-out)). Composer also redacts values that look like credentials: a value of eight characters or more under a name that looks like a credential, in the app configuration or the process environment, bearer and basic tokens, JWTs, a password in a URL and `token=` pairs. **The need lives in core and the source in the target.** Core's `secretSource(payload, { label, secretsKey })` builds a source from a target's payload, and never reads the payload. `secretsKey` is the app secrets key that holds the secret's value; Composer redacts the value under it, and `prisma dev` hands it to its second process. Prisma Cloud's `envSecret('NAME')` validates the name, brands the payload, and passes `secretsKey: NAME`. A different target could ship `vaultSecret({ path, key })` with no change to core. @@ -43,7 +43,7 @@ At boot the framework swaps each pointer for a redacting box over the named vari **Why names cross the boundary and values do not.** The deploy machine, the generated deploy program and deploy state are all things we inspect, diff and sometimes print; none was built to hold a secret. Rather than teach each to guard a value, the framework carries the platform variable's name and lets the platform inject the value into the running instance. -**Why Composer also redacts by value.** A secret's value still passes through Composer's process: preflight reads it to fill a missing variable, and `prisma dev` hands it to local services. An adapter's error, the deploy engine's failure cause or a cause chain could carry it. So Composer replaces every value it knows a secret holds, wherever it prints. It cannot replace a secret it was never told about, which is why the set is the values the graph's `envSecret()`s name. The app configuration and the app secrets hold values Composer may carry, and they usually hold the whole shell and every `.env` value; treating every value in them as a secret would blank every `1` and `true` in the output and refuse every plain value that equals one. +**Why Composer also redacts by value.** A secret's value still passes through Composer's process: `prisma dev` hands it to local services, and a deploying shell often holds the production value of a secret it deploys. An adapter's error, the deploy engine's failure cause or a cause chain could carry it. So Composer replaces every value it knows a secret holds, wherever it prints. It cannot replace a secret it was never told about, which is why the set is the values under the secrets keys of the graph's secret sources. The app configuration and the app secrets hold values Composer may carry, and they usually hold the whole shell and every `.env` value; treating every value in them as a secret would blank every `1` and `true` in the output and refuse every plain value that equals one. **Why the application binds the name.** If a reusable module hard-coded the variable it reads, every application using it would be forced onto that one name. A module that takes a `secretString()` field and lets its caller bind it stays reusable. @@ -54,7 +54,7 @@ At boot the framework swaps each pointer for a redacting box over the named vari - Every framework-written variable carries a reserved `COMPOSER_` prefix, so it can never overwrite a variable the user provisioned. - A secret field is required unless the schema makes it optional; presence is the schema's call. - Rotation is: change the value on the platform, then redeploy. A running instance's environment is frozen when the instance is created. -- Two input fields that normalize to the same platform variable name are a deploy error, so a secret and a generated param never share a variable. +- A secret and a generated param never share a variable: a generated variable's name starts with `COMPOSER_`, which `envSecret()` refuses. Two fields bound to `envSecret()` with one name read one variable. - A name may not be read both by `envParam()` and by `envSecret()`: Load refuses it, because the deploy hands its second process every plain value it reads, in a file. - A short secret is redacted at any length, so a one-character secret blanks that character in everything Composer prints. @@ -62,8 +62,8 @@ At boot the framework swaps each pointer for a redacting box over the named vari - **A forwardable `secret()` slot on modules**, which this decision first described as the need a module declares and its parent binds. Replaced by a `secretString()` field of the module's `input` ([ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md)). - **A `resolve()` function that receives the secret's source**, which an earlier version of the values decision allowed. Rejected: a function cannot compute anything useful from a value the deploy does not have. -- **Filling a missing variable from the deploying shell only.** Replaced by the app secrets, whose default, `[environmentVariables()]`, is the shell. -- **Redacting only values of eight characters or more.** Rejected for known secrets: a secret Composer knows is redacted at any length. The minimum stays for values Composer only guesses are credentials from their names, which is the existing behaviour this decision keeps for now. +- **Copying a missing variable at deploy, implicitly, from the app secrets or the deploying shell.** Rejected: with `.env` read by every environment, it is how a developer's local value reaches production ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). +- **Redacting only values of eight characters or more.** Rejected for known secrets: a secret Composer knows is redacted at any length. The minimum applies only to values Composer guesses are credentials from their names. ## Related @@ -71,4 +71,5 @@ At boot the framework swaps each pointer for a redacting box over the named vari - [ADR-0030](ADR-0030-rpc-callers-verified-with-an-auto-provisioned-service-key.md) — a generated param, which is config, not a secret. - [ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md) and [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — module input, values, and the adapters. - [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md) — how `prisma dev` reads secrets. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md) — platform variables: names, the marker, boot and preflight. - [config-params.md](../10-domains/config-params.md). diff --git a/docs/design/90-decisions/ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md b/docs/design/90-decisions/ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md index adcfc23af..b47b5aec8 100644 --- a/docs/design/90-decisions/ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md +++ b/docs/design/90-decisions/ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md @@ -16,15 +16,15 @@ const { appOrigin } = web.config(); // string ## Decision -A service's per-key param can be bound at `provision()` to a literal or to a target-owned source such as `envParam('NAME')`, instead of only carrying an authoring-time `default`. An env-sourced param is the plain sibling of [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md)'s secret: the same name-carried source, the same per-environment platform variable, the same preflight, but read back unredacted and validated by the param's own schema. +A service's per-key param can be bound at `provision()` to a literal or to a target-owned source such as `envParam('NAME')`, instead of only carrying an authoring-time `default`. An env-sourced param is the plain sibling of [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md)'s secret: the same name-carried source, the same per-environment platform variable that the operator sets, the same preflight check, but read back unredacted and validated by the param's own schema. `envParam('NAME')` returns a value, typed `Value`, that is also the source ([ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)). It reads `NAME` from the app configuration, what the `composer` section's `config` adapters return. Where it sits decides when: -- **In a service's per-key params**, the deploy writes a pointer row naming the platform variable. Preflight fills a missing variable from the app configuration. The service reads the variable at boot. -- **In a service's `input`**, the deploy writes the text into the service's input document. An unset or empty value leaves the key out, and the schema decides. +- **In a service's per-key params**, the deploy writes a pointer row naming the platform variable, which the operator sets on the platform. The service reads the variable at boot. A variable with no value gives the param its default, or leaves it out when the param is optional; boot fails only for a required param. +- **In a service's `input`**, the deploy writes the text into the service's input document. An unset or empty value leaves the key out, and the schema decides. This use is deprecated: a field of the root's input replaces it, and `envParam()`'s JSDoc carries `@deprecated`. - **In a module's `input` or as a `resolve()` dependency**, Load reads the text as the value's real value. An unset or empty value has none: in a binding the key is left out, and listed in `resolve()` it fails the Load, naming the variable. -The value must be text; anything else fails the Load, naming the key and the kind of value, before anything is created. Core knows a source only as an opaque payload; `paramSource(payload, { label, configurationKey, redacted })` is the constructor a target calls. `configurationKey` is the app configuration key whose text is the source's real value at deploy; `redacted` makes a source known only at boot a secret pointer rather than a plain one. Prisma Cloud's `envParam('NAME')` passes `configurationKey: NAME`. +The name follows the platform variable name rules of [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md): upper case, and not a reserved name. In a binding, the value must be text; anything else fails the Load, naming the key and the kind of value, before anything is created. Core knows a source only as an opaque payload; `paramSource(payload, { label, configurationKey, redacted })` is the constructor a target calls. `configurationKey` is the app configuration key whose text is the source's real value at deploy; `redacted` makes a source known only at boot a secret pointer rather than a plain one. Prisma Cloud's `envParam('NAME')` passes `configurationKey: NAME`. The deploy reads the app configuration once, in its first process, and hands its second process the text of every configuration key the graph reads, in the app configuration file ([ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)). @@ -36,18 +36,18 @@ The deploy reads the app configuration once, in its first process, and hands its **The wire: a pointer, told apart by shape.** An env-sourced param's stored row carries the platform variable's name, not a value. It shares its key, `COMPOSER_
_`, with a literal-bound param, so writer and reader tell them apart by the stored value. Literal values are JSON-encoded; a pointer row is `@composer-param-pointer:`. No `JSON.stringify` output begins with `@`, so the two never collide. -**Boot: double lookup, then the schema, on the raw string.** Boot reads the pointer for the name, reads that platform variable, and hands the raw string to the param's own schema. A `number()` param bound to `envParam` fails at boot with the existing invalid-value error, because the variable is a string. An unset platform variable is a boot error naming the param, the pointer key and the variable. An empty string is a value, which reaches the schema like any other. +**Boot reads the pointer, then the variable, then runs the schema on the raw string.** Boot reads the pointer for the name, reads that platform variable, and hands the raw string to the param's own schema. A `number()` param bound to `envParam` fails at boot with the existing invalid-value error, because the variable is a string. A variable that is unset or empty has no value, as the platform cannot store an empty one: the param takes its default, or is left out when it is optional. A required param fails boot with `The param "" of this service reads the platform variable "", which has no value. Set it with prisma project env add.` -**Preflight covers env-sourced params as it covers secrets.** Before anything is provisioned, the deploy checks that every env-sourced param's platform name exists for the environment, fills a missing one from the app configuration with one write-only call, and fails early, listing the names, when the app configuration lacks it too. +**Preflight covers env-sourced params as it covers secrets, and only checks them.** Before Alchemy runs, deploy preflight checks whether each per-key param's platform variable has a row the environment sees. A missing variable fails the deploy when the param has no default and is not optional, naming the variable, the service, the param and the commands that set it. Otherwise it is listed and the deploy goes on ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). Under `prisma dev` the local target gives the service the value from the app configuration, under the same rule. ## Consequences - A provision-time literal is schema-validated when config is built, and beats the `default`. - A required param that nothing binds fails the deploy loudly. - Env-sourced params are strings at the boundary. Decoding before the schema, per ADR-0019, is an additive change if a real consumer needs it. -- A value change needs a redeploy: a compute version snapshots its environment at creation. +- A value change needs a redeploy: a compute version snapshots its environment at creation. The deployment's triggers carry the variable's `updatedAt`, so a deploy after a change replaces exactly the services that read it. - The `@composer-param-pointer:` prefix is reserved only in raw stored-row space, which literals never occupy. -- A name may not be read both by `envParam()` and by `envSecret()`; Load refuses it. +- A name may not be read both by `envParam()` and by `envSecret()`; Load refuses it. A name may not be used both by `envParam()` and by a field of the root's input; the Prisma Cloud extension's graph check refuses it. ## Alternatives considered @@ -65,4 +65,5 @@ The deploy reads the app configuration once, in its first process, and hands its - [ADR-0031](ADR-0031-provisioned-param-values-are-a-need-resolved-through-a-target-registry.md) — framework-minted param values, which live in deploy state. - [ADR-0018](ADR-0018-config-params-carry-a-caller-owned-schema.md) and [ADR-0019](ADR-0019-the-target-owns-config-serialization.md) — schema on the declaration, wire owned by the target. - [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — values and the app configuration. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md) — platform variables: names, boot and preflight. - [config-params.md](../10-domains/config-params.md). diff --git a/docs/design/90-decisions/ADR-0042-service-input-is-one-standard-schema.md b/docs/design/90-decisions/ADR-0042-service-input-is-one-standard-schema.md index a632380a4..c80faad19 100644 --- a/docs/design/90-decisions/ADR-0042-service-input-is-one-standard-schema.md +++ b/docs/design/90-decisions/ADR-0042-service-input-is-one-standard-schema.md @@ -39,7 +39,7 @@ A compute service declares all of its input, plain values and secrets together, - **The binding** mirrors the schema's shape. Each leaf is a literal, a value such as a `resolve()` result or a module's input field ([ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md)), or a source: `envParam()`, `envSecret()`, `generatedParam()` or `serviceOrigin()`, each of which is a value too. - **The binding is typed from the schema's input type**, with the same mapping a module's binding uses ([ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md)). So a wrong literal, an `envSecret()` in a text field, or a `serviceOrigin()` in a plain field fails to compile. - **The deploy checks the binding with the schema** before anything is created, with each value known only at boot in place as its pointer, and writes the schema's output into one input document. -- **At boot** `service.input()` reads that document, puts each secret in a redacting box, checks the result with the schema again, and returns it. +- **At boot** `service.input()` reads that document, leaves out each field whose platform variable has no value, puts each secret in a redacting box, checks the result with the schema again, and returns it. ## The binding's type @@ -59,21 +59,28 @@ A schema may return a copy of a pointer: ArkType copies its input when a transfo ## The input document -The deploy writes the schema's output, with defaults applied, into one environment row per service, `COMPOSER_
_INPUT`. Each pointer is written as a marker. A secret pointer from `envSecret()` becomes `{"$secret": ""}`, naming the operator's platform variable, which preflight checks exists. A `generatedParam()` and a `serviceOrigin()` become `{"$generated": "", "redacted": }`, naming a variable in Composer's reserved `COMPOSER_` namespace that the deploy fills: with a value it generates once and keeps in deploy state, or with the other service's public origin. A marker records `redacted` because boot cannot ask the schema which fields are secret. +The deploy writes the schema's output, with defaults applied, into one environment row per service, `COMPOSER_
_INPUT`. Each pointer is written as a marker. A secret pointer from `envSecret()` becomes `{"$platform": {"name": "", "secret": true}}`, naming a platform variable the operator sets on the platform ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). A `generatedParam()` and a `serviceOrigin()` become `{"$generated": "", "redacted": }`, naming a variable in Composer's reserved `COMPOSER_` namespace that the deploy writes: with a value it generates once and keeps in deploy state, or with the other service's public origin. A `$platform` marker records `secret`, and a `$generated` marker records `redacted`, because boot cannot ask the schema which fields are secret. The marker format and the code that reads it live in `@internal/input-document`, which the serializer, boot and the local target share. ```json { "stripeEnabled": true, "returnUrl": { "$generated": "COMPOSER_SHOP_CHAT_RETURNURL_GENERATED", "redacted": false }, - "stripeSecretKey": { "$secret": "STRIPE_SECRET_KEY" } + "stripeSecretKey": { "$platform": { "name": "STRIPE_SECRET_KEY", "secret": true } } } ``` -No secret's value and no generated value appears in the document, so the deploy report prints it verbatim, with every key that resolved absent. A user key spelled like a marker, such as `$secret` or `$generated`, gains one more leading `$` when written and loses it when read, so only Composer can place a marker. Every variable a service's markers name must be distinct once normalised, or the deploy fails naming both input paths. The document is JSON, so the schema's output must be plain objects, arrays, text, numbers, booleans, null and secret boxes. A schema that outputs a `Date`, a `BigInt` or another class instance is outside this contract; keep the field as text and convert it in the service. +No secret's value and no generated value appears in the document. The deploy report prints it with each marker shown as `$platform NAME` or `$generated NAME`, with every key that resolved absent. A user key spelled like a marker, `$platform` or `$generated`, gains one more leading `$` when written and loses it when read, so only Composer can place a marker. The `$generated` variables of one service must be distinct once upper-cased, or the deploy fails naming both input paths, because Composer writes each one. Two `$platform` markers for one name are two reads of one variable and are allowed. The document is JSON, so the schema's output must be plain objects, arrays, text, numbers, booleans, null and secret boxes. A schema that outputs a `Date`, a `BigInt` or another class instance is outside this contract; keep the field as text and convert it in the service. -At boot `service.input()` reads the one row, replaces each `$secret` marker with a box over the named variable and each `$generated` marker with a box or text as its `redacted` says, checks the result with the schema, and returns the schema's output. Its type is the schema's output type with every pointer removed, at any depth, because at boot each pointer has been replaced by the variable's text: a wrapped field reads as its inner type, and a `secretString()` field as `SecretString`. `mockService()`'s input double has the same type. +At boot `service.input()` reads the one row and replaces each marker in four steps: -An `envParam()` field whose variable is unset or empty is a key left out, and whether that is allowed is the schema's call: an optional field, a union member, or an error with the schema's message. So an optional secret, or a field that exists only when another field is set, is an ordinary schema, with no flag of Composer's. +1. A `$platform` marker whose variable has no value, unset or empty, leaves its key out of its object. A marker in an array position cannot be left out, so boot fails naming the variable and the path. +2. A `$platform` marker whose variable has text takes the text. +3. A marker with `secret: true` puts the text in a secret box. A `$generated` marker becomes a box or text as its `redacted` says. +4. The schema checks the result. When it fails and step 1 left out at least one variable, the error ends with the names of those variables, so the operator knows what to set. + +`service.input()` returns the schema's output. Its type is the schema's output type with every pointer removed, at any depth, because at boot each pointer has been replaced by the variable's text: a wrapped field reads as its inner type, and a `secretString()` field as `SecretString`. `mockService()`'s input double has the same type. + +An `envParam()` field whose variable is unset or empty in the app configuration is a key left out at deploy, and an `envSecret()` field whose platform variable is unset or empty is a key left out at boot. Whether that is allowed is the schema's call: an optional field, a union member, or an error with the schema's message. So an optional secret, or a field that exists only when another field is set, is an ordinary schema, with no flag of Composer's. Empty means unset because the platform cannot store an empty value. ## Reasoning @@ -95,7 +102,7 @@ An `envParam()` field whose variable is unset or empty is a key left out, and wh - The type `InputBinding` is shared by modules and services. A binding whose shape or kind the schema's input type refuses fails to compile: a wrong literal type, a missing or undeclared key, a source in the wrong kind of field, or a missing `input`. A refinement the type cannot express, such as a URL format, is checked by the deploy, as is a binding the compiler cannot see: a schema without type information, or a cast. - Errors about a binding's data are the schema library's own messages, at compile time as a type error and at deploy as the library's issue text. - A schema that copies its input cannot hold a secret field in ArkType; the deploy refuses it and names the field. -- The deploy report prints each service's input document and every key that resolved absent. +- The deploy report prints each service's input document, each marker as `$platform NAME` or `$generated NAME`, and every key that resolved absent. - Composer's own per-service values, such as another service's URL wired as a dependency, still travel on per-key rows ([ADR-0031](ADR-0031-provisioned-param-values-are-a-need-resolved-through-a-target-registry.md)). Moving them into a document like this one is a separate decision. ## Alternatives considered @@ -112,6 +119,7 @@ An `envParam()` field whose variable is unset or empty is a key left out, and wh - [ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md): a module declares its input the same way, and its binding has the same type. - [ADR-0016](ADR-0016-a-module-has-the-same-boundary-as-a-service.md): a module has the same boundary as a service. - [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md): values, runtime values and pointers, and the app configuration `envParam()` reads. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md): platform variables, the `$platform` marker's rules, and the deploy preflight. - [ADR-0021](ADR-0021-params-are-read-through-config-not-load.md), superseded: `config()` is replaced by `service.input()`; dependencies still read through `load()`. - [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md), partly superseded: `envSecret()` survives as a binding leaf; the `secrets()` read and per-slot pointer rows are replaced. - [ADR-0032](ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md), partly superseded: binding at `provision()` and `envParam()` survive; the `config()` read and per-key pointer rows are replaced. diff --git a/docs/design/90-decisions/ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md b/docs/design/90-decisions/ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md index 982d4ef34..31c9e2ded 100644 --- a/docs/design/90-decisions/ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md +++ b/docs/design/90-decisions/ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md @@ -11,8 +11,8 @@ composer({ /* … */ config: [dotenv(), environmentVariables()], secrets: [doten $ prisma dev prisma dev cannot start: -required secrets are not set: - - STRIPE_SECRET_KEY (required by service "billing": stripeKey must be present (was missing)) +1 platform variable a service needs has no value: + - STRIPE_SECRET_KEY (required by service "shop.billing": stripeKey must be present (was missing)) Set each missing variable where an adapter in prisma.config.ts reads it: the `secrets` list of the `composer` section for a secret, the `config` list for an env-sourced param. environmentVariables() reads the shell you run `prisma dev` from; dotenv() reads .env, .env.local, .env.dev and .env.dev.local beside prisma.config.ts. ``` @@ -23,15 +23,15 @@ Under `prisma dev`, a secret's value comes only from the app secrets, and an env - **A required secret that is unset or empty stops `prisma dev`.** The error lists every missing variable, the service that needs it, the schema's message, and where to set it. Empty counts as unset. - **An optional secret that is unset or empty reaches the service as absent.** Its input has no value for that field, and the schema decides what that means, as it does for an unset `envParam` leaf ([ADR-0042](ADR-0042-service-input-is-one-standard-schema.md)). -- **Required or optional is decided by validation, not by reading the schema.** For each service the dev preflight resolves the input binding with every unset secret left out and calls the schema's `validate`. If that passes, the missing secrets were optional. If it fails, the preflight checks each unset secret alone and names as required only the secrets whose absence alone fails. When no single secret fails alone, the error says the service needs some of them, and calls none of them required. -- **Env-sourced params get the same sourcing.** An unset or empty env-sourced param is always an error. Like secrets, params are read fresh each run and fingerprinted, so a changed value redeploys the services that read it. -- **The second process receives the values; it reads none.** `prisma dev` runs Alchemy in a second process at start and at each rebuild. That process runs no adapter. It reads what Load read from the app configuration, and the first process's node addresses, from the app configuration file, `/app-configuration-.json`, readable only by its owner, which `prisma dev` writes before each converge and deletes as soon as that converge ends. It receives the named secrets, each app secret an `envSecret()` in the graph names, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS`, in memory; no secret is written to a file for it. +- **Required or optional is decided by validation, not by reading the schema.** For each service the dev preflight resolves the input binding with every unset secret left out and calls the schema's `validate`. If that passes, the missing secrets were optional. If it fails, the preflight checks each unset secret alone and names as required only the secrets whose absence alone fails. When no single secret fails alone, the error says the service needs at least one of them, and calls none of them required. A secret an array position reads is required without a check, because an array position cannot be left out. The deploy preflight decides required or optional for a platform variable with no row by the same method and the same code ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). +- **Env-sourced params get the same sourcing.** A per-key param bound to `envParam()` reads its value from the app configuration. When the value is unset or empty, the param takes its default, or is left out when it is optional; it stops `prisma dev` only when it has no default and is not optional, naming the variable, the service and the param. A value that is not text stops `prisma dev`, naming the variable, the service and the kind of value. Like secrets, params are read fresh each run and fingerprinted, so a changed value redeploys the services that read it. +- **The second process receives the values; it reads none.** `prisma dev` runs Alchemy in a second process at start and at each rebuild. That process runs no adapter. It reads what Load read from the app configuration, and the first process's node addresses, from the app configuration file, `/app-configuration-.json`, readable only by its owner, which `prisma dev` writes before each converge and deletes as soon as that converge ends. It receives the named secrets, the app secrets under the secrets keys of the graph's secret sources, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS`, in memory; no secret is written to a file for it. - **The dev preflight hands the second process a payload.** It travels on the preflight transport that carries deploy's rotation timestamps (`PRISMA_COMPOSER_PREFLIGHT_`), with a `dev:` prefix that a deploy payload, always a JSON object, never starts with. It names the absent secrets and carries a fingerprint of the current value of each variable that a service's input or params point at, never the value. The serializer leaves an `envSecret` leaf out of the input document only when this payload names it absent. -- **Each service gets only the variables its own rows point at**: the `$secret` pointers in its input document and its param pointer rows. A service that runs a dev server ([ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md)) is given the same environment, and a changed value restarts it. +- **Each local service gets only the variables its own rows point at**: the `$platform` markers in its input document, at any depth, and its param pointer rows. This is a rule of the local target. On Prisma Cloud every deployment receives every platform variable of its class ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). A service that runs a dev server ([ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md)) is given the same environment, and a changed value restarts it. - **A changed value redeploys the services that read it.** Each `Deployment`'s triggers carry a `:updatedAt` member per variable its rows point at. On a hosted deploy it holds the platform variable's `updatedAt`. Under `prisma dev` it holds the fingerprint from the payload: an HMAC-SHA256 of the value under a random 32-byte key that `prisma dev` creates once in `.prisma-composer/dev//fingerprint.key` (mode 0600). Alchemy writes trigger members to its state as given, which is why the fingerprint is keyed: a bare hash of a short secret could be reversed by guessing. A lost key causes one redeploy, and `--fresh` deletes it. - **A changed adapter value pauses rebuilds.** When what the adapters return at a rebuild differs from what they returned at start, `prisma dev` keeps the values it started with and stops rebuilding until it is restarted, as it does after an edit to `prisma.config.ts`. - **Composer never reads a secret back from storage.** The `secrets.json` store in `.prisma-composer/dev/` is gone, and `prisma dev` deletes a leftover one when it starts. -- **Deploy is unchanged by this decision.** Deploy sends no dev payload, so the serializer leaves nothing out. +- **The deploy passes no secret values to the second process.** Its input document holds a marker for every secret, and boot leaves out a key whose variable has no value. ## Reasoning @@ -47,11 +47,11 @@ The second process needs the dev preflight's findings and the secret values, and - An app whose required secret is unset locally stops with an error naming the variable. Optional secrets that are unset arrive absent. - Changing a secret and running `prisma dev` again, without `--fresh`, restarts exactly the services whose rows point at it. -- A service receives only the secrets and env-sourced params its own input and params point at. Code that reads `process.env.X` directly, where `X` is bound by another service, gets nothing. -- **A value in `.env` or `.env.local` is read by every environment, including `prisma deploy` to `production`.** Deploy's preflight uploads any secret the target environment does not have yet, so a local-only value there becomes the hosted value. Put local-only values in `.env.dev` or `.env.dev.local`, which only the environment `dev` reads. A project that adopts `dotenv()` removes `import 'dotenv/config'` from `prisma.config.ts`, or `.env`'s values reach the shell's place in the order. +- Under `prisma dev`, a service receives only the secrets and env-sourced params its own input and params point at. Code that reads `process.env.X` directly, where `X` is bound by another service, gets nothing locally, and gets the value on Prisma Cloud. +- **A value in `.env` or `.env.local` is read by every environment, including `prisma deploy` to `production`.** The deploy does not copy a secret to the platform, but a plain value there reaches the root's input and every `envParam()` in every environment. Put local-only values in `.env.dev` or `.env.dev.local`, which only the environment `dev` reads. A project that adopts `dotenv()` removes `import 'dotenv/config'` from `prisma.config.ts`, or `.env`'s values reach the shell's place in the order. - Each environment's folder under `.prisma-composer/dev/` holds its own fingerprint key, mode 0600. Alchemy's local state holds HMAC fingerprints of secret values, never the values. - The compute emulator keeps a copy of each running service's environment, secret values included, in its `instances.json` (mode 0600, in the machine-wide emulator directory, outside the project), so it can restart the service after the emulator itself restarts. Composer never reads that copy back. This is recorded in `.drive/deferred.md`. -- Dev and deploy write different input documents for the same app when an optional secret is unset: dev leaves the field out, as ADR-0042 promises for every env-bound leaf, and deploy writes a `$secret` pointer and fails when the platform lacks the variable. The deferred deploy fix in `.drive/deferred.md` would make the two the same. +- Dev and deploy give a service the same input when an optional secret is unset: the field is absent. Dev's input document leaves the marker out, and deploy's holds it and boot leaves the key out. ## Alternatives considered @@ -70,4 +70,5 @@ The second process needs the dev preflight's findings and the secret values, and - [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — the input schema, the binding, and validation as the only judge of absence. - [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md) — the secrets model. - [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — the adapters, the app configuration and the app secrets. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md) — platform variables on Prisma Cloud, and the deploy preflight that uses this record's method. - [local-dev.md](../10-domains/local-dev.md) and [running-locally.md](../../guides/running-locally.md). diff --git a/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md b/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md index 40513e5f7..7a194ae23 100644 --- a/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md +++ b/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md @@ -39,11 +39,11 @@ prisma deploy --environment pr-42 # a preview prisma dev # the environment dev, on this machine ``` -`prisma deploy` reads `.env`, `.env.production` and their `.local` files, then the shell. The merged result is the root module's input, so `input.EMAIL_FROM` and `input.SITE_HOSTNAME` come from there. The `resolve()` picks the public URL's hostname from `SITE_HOSTNAME` if it is set, and otherwise from the environment's name. `STRIPE_SECRET_KEY` never enters the module: the service reads it from a platform variable at boot, and the deploy fills that variable from the secrets adapters if it does not exist yet. +`prisma deploy` reads `.env`, `.env.production` and their `.local` files, then the shell. The merged result is the root module's input, so `input.EMAIL_FROM` and `input.SITE_HOSTNAME` come from there. The `resolve()` picks the public URL's hostname from `SITE_HOSTNAME` if it is set, and otherwise from the environment's name. `STRIPE_SECRET_KEY` never enters the module: the service reads it from a platform variable at boot. The operator sets that variable on the platform; the deploy checks that it exists and does not write it. ## Decision -A module computes a value that depends on the environment, or on its inputs, with **`resolve()`**. Values that live outside the code reach the root module's `input` through **config adapters**, and secrets reach platform variables through **secrets adapters**, both named in the `composer` section of `prisma.config.ts`. Every environment has a name, and none is special. +A module computes a value that depends on the environment, or on its inputs, with **`resolve()`**. Values that live outside the code reach the root module's `input` through **config adapters**, named in the `composer` section of `prisma.config.ts`. The deploy does not copy an adapter's value to a platform variable, because an implicit copy can push a local value into a remote environment; for now the operator sets platform variables on the platform ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). **Secrets adapters**, named in the same section, supply secrets to `prisma dev` and tell Composer which values to redact. Every environment has a name, and none is special. All of this serves one requirement: **the graph's structure is the same whatever inputs Composer is given.** Values may differ per environment; nodes and edges may not. @@ -108,9 +108,9 @@ Some values belong to a file an ops team owns, or to a secrets manager, not to ` - An adapter receives `{ environment, configDirectory }`: the environment's name and the folder of the declaring `prisma.config.ts`. It returns an object, or a Promise of one. - The adapters of a list run one at a time, in order. Their results merge by top-level key, and a later adapter wins. A key whose value is `undefined` does not hide an earlier adapter's value. -- The merged result of the config adapters is **the app configuration**. "App" is what Composer calls the application, the outermost module, whose input it fills. It is the root's input, and `envParam()` reads from it. Deploy preflight also fills a missing platform variable for an `envParam()` in a service's per-key params from it. -- The merged result of the secrets adapters is **the app secrets**. Every value in it must be text. Deploy preflight and dev preflight fill a missing `envSecret()` platform variable from it, by name. The app secrets never reach Load or a schema. -- Preflight fills only a platform variable that does not exist yet. Editing a value in `.env.production` and deploying again does not change a variable the platform already holds. To change one, change it on the platform. +- The merged result of the config adapters is **the app configuration**. "App" is what Composer calls the application, the outermost module, whose input it fills. It is the root's input, and `envParam()` reads from it. Nothing fills a platform variable from it on a deploy; under `prisma dev` the local target gives a per-key `envParam()` param its value from it. +- The merged result of the secrets adapters is **the app secrets**. Every value in it must be text. Nothing fills a platform variable from it. Under `prisma dev` the local target gives each service the named secrets, the values under the secrets keys of the graph's secret sources, that its own markers name. On a deploy, Composer reads it only to know which values to redact. The app secrets never reach Load or a schema. +- The deploy does not copy local values to platform variables; the operator sets them on the platform ([ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md)). Editing a secret in `.env.production` and deploying again changes nothing on the platform. To change one, change it on the platform and deploy again. - Both lists default to **`[environmentVariables()]`**, so a project that writes neither reads the shell. A list the author writes replaces the default. To keep reading the shell, include `environmentVariables()`. - An adapter that throws fails the command with `CONFIG.ADAPTER_FAILED`. An adapter that returns something other than an object, or a secrets adapter that returns a value that is not text, fails it with `CONFIG.ADAPTER_INVALID`. - Composer reads its own credentials (`PRISMA_SERVICE_TOKEN`, `PRISMA_WORKSPACE_ID`, `PRISMA_REGION`) from the process environment, never from the adapters. @@ -132,11 +132,11 @@ The `dotenv` package defines no order among these files, and Next.js and Vite di A deploy runs Alchemy in a second process, which Loads the graph again. That process runs no adapter. The first process writes the **app configuration file** beside the generated stack file, `.prisma-composer/app-configuration--.json`, readable only by its owner and deleted when the deploy ends. It holds what Load read from the app configuration and the first process's node addresses. What Load read is the real value of each path of the root's input that the root's body passes on, and the text of each configuration key an `envParam()` reads. The second process refuses a graph whose node addresses differ from the first process's. So a `.env` file edited during the deploy, or an adapter that would return something else a second time, cannot change what is deployed. -The app configuration file holds no secrets. Load refuses, with `COMPOSE.GRAPH_INVALID`, a name that both an `envParam()` and an `envSecret()` read, and a root input field the root's body passes on when an `envSecret()` in the graph reads the same name. The CLI refuses to write the file, with `CONFIG.SECRET_AS_PLAIN_VALUE`, when any value in it contains a known secret: a value under a name an `envSecret()` in the graph reads, in the app secrets or the app configuration. A known secret of eight characters or more is refused anywhere in a value, such as the password in a URL. A shorter one is refused only when it equals the whole value, so a placeholder secret such as `dev` does not refuse `dev@example.com`. A number or a boolean is compared as its text. Both refusals name the field, never the value; the CLI's refusal also names the secrets key whose value matched. +The app configuration file holds no secrets. Load refuses, with `COMPOSE.GRAPH_INVALID`, a name that both an `envParam()` and an `envSecret()` read, and a root input field the root's body passes on when an `envSecret()` in the graph reads the same name. The CLI refuses to write the file, with `CONFIG.SECRET_AS_PLAIN_VALUE`, when any value in it contains a known secret: a value under a secrets key of the graph's secret sources, in the app secrets or the app configuration. A known secret of eight characters or more is refused anywhere in a value, such as the password in a URL. A shorter one is refused only when it equals the whole value, so a placeholder secret such as `dev` does not refuse `dev@example.com`. A number or a boolean is compared as its text. Both refusals name the field, never the value; the CLI's refusal also names the secrets key whose value matched. `destroy` runs no `secrets` adapter, so its check compares against the values under those names in the app configuration only. On `destroy`, which reads no secrets, a secret only the `secrets` list returns is not known, so the file check cannot refuse it. That is a known limit; a deploy of the same configuration refuses first. Load's refusal of a root field named like an `envSecret()` still applies on destroy. -`prisma dev`'s second process receives the app configuration file the same way. `prisma dev` writes it to the dev environment's directory before each converge and deletes it as soon as that converge ends. Under `prisma dev` only, the second process also receives the named secrets, each app secret an `envSecret()` in the graph names, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS`, in memory. So the handover is one file, plus one variable under `prisma dev`. +`prisma dev`'s second process receives the app configuration file the same way. `prisma dev` writes it to the dev environment's directory before each converge and deletes it as soon as that converge ends. Under `prisma dev` only, the second process also receives the named secrets, the app secrets under the secrets keys of the graph's secret sources, in one environment variable, `PRISMA_COMPOSER_NAMED_SECRETS`, in memory. So the handover is one file, plus one variable under `prisma dev`. The CLI also deletes the file when its process exits, so a run stopped by a second Ctrl-C leaves none. A process killed outright cannot delete its file. So at the start of each deploy, destroy and `prisma dev`, the CLI deletes the app configuration files under `.prisma-composer/` and each dev environment directory whose writing process is not running, by the process id in the file name, and that are older than ten minutes. The age check keeps the file of a run in a container or on another machine that shares the folder, whose process id means nothing here. @@ -153,7 +153,7 @@ A root's fields stay plain. The app configuration never holds a pointer, so wrap A secret's value enters no file Composer writes for a deploy, no deploy state, no deploy report and no log line. - **Pointers.** An `envSecret()` reaches the service's input document as a pointer. The service reads the variable at boot. No `resolve()` sees a secret, because none can list one. -- **Redaction.** Everywhere Composer prints, including errors, events, adapter failures, a thrown defect's cause chain and the second process's failure cause, it redacts one set of known secrets: the values under the names an `envSecret()` in the graph reads, in the app secrets and in the app configuration. The app configuration and the app secrets hold values Composer may carry; nothing else in them is a known secret. `prisma dev` compares the whole app secrets between rebuilds to notice a change, which reads them without making them known secrets. A plain value that shares a `.env` file with a secret is not redacted, and under `secrets: [dotenv(), environmentVariables()]` neither is the shell's `SHLVL=1`. Composer also keeps the redaction it already did, which is not part of this decision: a value of eight characters or more under a name that looks like a credential, in the app configuration or the process environment, and token shapes such as JWTs. A known secret is redacted at any length. When two known secrets overlap in the text, the redactor hides every character either one covers. Nothing is redacted by the known secrets before Load has named them: an adapter that throws, and a root schema failure that echoes a field's value, are redacted only by the credential-looking rule. That is a known limit: the names come from the graph. +- **Redaction.** Everywhere Composer prints, including errors, events, adapter failures, a thrown defect's cause chain and the second process's failure cause, it redacts one set of known secrets: the values under the secrets keys of the graph's secret sources, in the app secrets and in the app configuration. The app configuration and the app secrets hold values Composer may carry; nothing else in them is a known secret. `prisma dev` compares the whole app secrets between rebuilds to notice a change, which reads them without making them known secrets. A plain value that shares a `.env` file with a secret is not redacted, and under `secrets: [dotenv(), environmentVariables()]` neither is the shell's `SHLVL=1`. Composer also redacts values that look like credentials: a value of eight characters or more under a name that looks like a credential, in the app configuration or the process environment, and token shapes such as JWTs. A known secret is redacted at any length. When two known secrets overlap in the text, the redactor hides every character either one covers. Nothing is redacted by the known secrets before Load has named them: an adapter that throws, and a root schema failure that echoes a field's value, are redacted only by the credential-looking rule. That is a known limit: the names come from the graph. - **Core knows each secret's name.** A target's secret source tells core which app secrets key holds its value: `secretSource()` takes `secretsKey`, and `envSecret('NAME')` passes `NAME`. That is how Composer knows what to redact, and which secrets `prisma dev` hands its second process. - **Alchemy's output.** Under the CLI, the command engine gives the second process the terminal, so Composer never sees Alchemy's output and does not redact it, for now. Redacting it needed a change to the engine (prisma/prisma-cli#383), which was dropped: Composer will soon run Alchemy through its programmatic API, in its own process, which lets Composer redact that output. That work must redact Alchemy's output itself. A programmatic host's default runner, `spawnAlchemy`, pipes the second process's output through the stream redactor, so there it is redacted and prints plain lines without colour or a live display. The stream redactor's cost grows with the square of the longest secret's length; that is a known limit, which matters only for secrets of many kilobytes. Under both, the redactor covers everything Composer itself prints, including the second process's failure cause. - A schema's refusal of a `secretString()` field says what kind of value it got, never the value. @@ -175,7 +175,7 @@ There is one exception, inherited from `dotenv-flow`: `dotenv()` skips `.env.loc - **`@prisma/composer` exports `resolve()`, `Value`, `RuntimeValue` and `Pointer`**, target-neutral, with `isSecretPointer()` and `isPlainPointer()` for authors of schema helpers. Every node parameter and input binding that carries data accepts a `Value` of its type; `name`, `id`, `deps` and `target` do not. The module body receives `environment` beside `input` and `provision`. - **The `composer` section has `config` and `secrets`**, each a list of adapters defaulting to `[environmentVariables()]` ([ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md)). `@prisma/composer/config` exports `dotenv()` and `environmentVariables()`. The errors decision gains `CONFIG.ADAPTER_FAILED` and `CONFIG.ADAPTER_INVALID`. - **`prisma deploy` and `prisma dev` take `--environment`**, defaulting to `production` and `dev` ([ADR-0059](ADR-0059-a-deployment-environment-is-called-an-environment.md), [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md)). `destroy` takes an environment name with no default. -- **The app configuration is plain.** A secret belongs with `envSecret()`, whose platform variable the secrets adapters fill. The root's schema refuses a plain value where it expects `secretString()`. +- **The app configuration is plain.** A secret belongs with `envSecret()`, whose platform variable the operator sets on the platform. The root's schema refuses a plain value where it expects `secretString()`. - **`dotenv()` partly reverses the dev secrets decision's rejection of loading `.env` in Composer itself** ([ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md)). Composer ships the loader, but the user still names it in `prisma.config.ts`, so the user still chooses it. - **This restates the environment-resolution decision** ([ADR-0024](ADR-0024-a-stage-is-a-deploy-time-environment-resolved-to-project-and-branch.md)): its statement that the environment never appears in the authored topology becomes the requirement that the graph is the same whatever inputs Composer is given. @@ -208,4 +208,5 @@ There is one exception, inherited from `dotenv-flow`: `dotenv()` skips `.env.loc - [ADR-0024](ADR-0024-a-stage-is-a-deploy-time-environment-resolved-to-project-and-branch.md) and [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md): environment resolution and the `deploy` and `destroy` surfaces. - [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md): the `composer` section this extends. - [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md) and [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md): secrets, and how `prisma dev` reads them. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md): platform variables, which the operator sets and the deploy does not copy from local values. - [local-dev.md](../10-domains/local-dev.md) and [config-params.md](../10-domains/config-params.md). diff --git a/docs/design/90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md b/docs/design/90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md new file mode 100644 index 000000000..da1244bc6 --- /dev/null +++ b/docs/design/90-decisions/ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md @@ -0,0 +1,166 @@ +# ADR-0064: The deploy does not copy local values to platform variables + +```ts +// module.ts +export default appRoot('shop', ({ provision }) => { + provision(emailService(), { + input: { from: 'shop@example.com', apiKey: envSecret('EMAIL_API_KEY') }, + }); +}); +``` + +```sh +$ prisma deploy --environment staging +✔ check environment variables +Platform variables with no value: + - EMAIL_API_KEY (read by service "shop.email", field "apiKey") + prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview + prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview --branch 'staging' +``` + +## Decision + +A **platform variable** is an environment variable on the platform that a service reads at boot. `envSecret('NAME')` and a per-key `envParam('NAME')` param name one. + +The deploy does not copy a platform variable's value from the developer's local environment: the `secrets` and `config` adapters, by default `.env` files and the shell. An implicit copy can push a local value into a remote environment by accident. For now the operator sets each value on Prisma Cloud with `prisma project env add`, and the deploy creates, changes and deletes no platform variable. Preflight checks only that every variable a service needs has a value. This is not a rule that the deploy may never set a variable. An explicit way to set variables through `deploy`, where the operator says which values go to which environment, is expected later (see Future work). + +Destroying the production environment removes the Project, and with it every variable set on it. Composer records which service reads which variable, checks at deploy that every variable a service needs has a value, lists the ones with none, and reads the value only inside the running service. The rows Composer does write are its own: the `COMPOSER_…` rows of each service, and the create-only `DATABASE_URL` and `DATABASE_URL_POOLED` placeholders of [ADR-0048](ADR-0048-prisma-cloud-resources-come-from-the-upstream-alchemy-provider.md). An app can read none of those names. + +### What Prisma Cloud does + +- **A variable is a row in a project.** A row has `projectId`, `class` (`production` or `preview`), an optional `branchId`, a `key` and a `value`. A row with no `branchId` is the class's shared value. A row with one is that preview branch's override. +- **A deployment receives the rows of its class, with its branch's overrides on top.** The platform copies them into the deployment when it creates it and never reads them again, so a value set later reaches a service only at its next deployment. There is no service scope: every deployment receives every row of its class, its own and other services' alike. The list filter accepts only `projectId`, `class`, `key` and `branchId`. +- **A key is upper case or it does not exist.** The platform refuses a key that does not match `[A-Z_][A-Z0-9_]*`, on write and in the list filter. +- **A value is never empty.** The platform refuses an empty value, so a variable is either present with text or absent. +- **A read returns metadata, never a value.** It returns `updatedAt`, and `isManagedBySystem` for a row the platform owns. +- **The operator sets a row with the `prisma` CLI.** `prisma project env add 'NAME=' --project --role production` sets the production row. `--role preview` sets the preview-wide row that every preview branch shares. `--role preview --branch ` sets one branch's override. `update`, `list` and `delete` take the same scopes. +- **A process that exits at boot is reported running.** The deployment is started and promoted as soon as it is created. The platform reports it `running`, `prisma deploy` succeeds, and the crashing deployment is promoted. Each request to the service boots it once and gets a 502. The boot error is only in the logs: `prisma service logs --project `. + +### Names + +`envSecret()` and `envParam()` check the name they are given, and Load fails with a message naming the rule. + +- A name matches `^[A-Z][A-Z0-9_]*$`: an upper-case letter, then upper-case letters, digits and underscores. The platform also accepts a leading underscore; Composer refuses it (see Reasoning). +- These names are reserved: any name starting with `COMPOSER_`, which are Composer's own rows; `DATABASE_URL` and `DATABASE_URL_POOLED`, which Prisma Cloud sets and manages; `PORT`, which boot sets to the service's port; `PATH` and `HOME`, which `prisma dev` sets from the developer's shell. `HOST` is allowed. +- **One name has one meaning.** Load refuses a name that `envParam()` or a root input field reads from the app configuration when an `envSecret()` names it as a secret. The Prisma Cloud extension's graph check refuses a name that both an `envParam()` and a field of the root's input use. The error names both uses. Two `envSecret('X')`, or two `envParam('X')`, read one variable and are allowed. + +### The graph check + +`ExtensionDescriptor` has an optional `checkGraph(graph)`. The CLI calls every extension's `checkGraph` right after Load and before the registry coverage check. So it runs in `prisma deploy`, in `prisma dev` at start and at every rebuild, and in `destroy`. A command that reads only the app's name, such as `log`, does not run Load and does not run the check. A check refuses a graph by throwing core's `GraphCheckError`, which fails the command with `COMPOSE.GRAPH_INVALID` and its message, before anything is created and before any dev service starts. Any other throw from the hook is a bug and reaches the user as one ([ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md)). The check sees the loaded graph only, never the app configuration or the app secrets. + +### The input document + +Each place in a service's input document that reads a platform variable holds a `$platform` marker: + +```json +{ "from": "shop@example.com", "apiKey": { "$platform": { "name": "EMAIL_API_KEY", "secret": true } } } +``` + +- `name` is the variable. `secret` is `true` for `envSecret()`. +- A `$generated` marker names a variable Composer writes itself, for `generatedParam()` and `serviceOrigin()`. Two `$generated` markers in one document must name different variables, compared without case. Two `$platform` markers for one name are two reads of one variable and are allowed. +- A user key spelled like a marker (`$platform` or `$generated`) gains one leading `$` when written and loses it when read, so only Composer can place a marker. +- The deploy report shows each marker as `$platform NAME` or `$generated NAME`. +- The marker format, the code that finds every marker at any depth, boot's handling of a marker, the names of Composer's own `COMPOSER_…` rows, and the rows that tell boot which platform variable each per-key param reads live in one package, `@internal/input-document`. The serializer, boot and the local target all use it. + +### Boot + +`service.input()` reads the input document in four steps: + +1. **A marker whose variable has no value** leaves its key out of its object. No value means `process.env[name]` is undefined or empty. A marker in an array position cannot be left out: boot fails, naming the variable and the path. +2. **A marker whose variable has text** takes the text. +3. **A marker with `secret: true`** puts the text in a secret box. +4. **The service's schema validates the whole object.** When it fails and step 1 left out at least one variable, the error ends with `(platform variables with no value: X, Y)`. + +A per-key `envParam()` param whose variable has no value takes the param's default, or is left out when the param is optional. Boot fails only for a required one, with: `The param "" of this service reads the platform variable "", which has no value. Set it with prisma project env add.` + +Empty means unset everywhere: at boot, for a per-key param, and under `prisma dev`. + +### Deploy preflight + +Preflight runs after the deploy has resolved the app's Project and Branch, and before Alchemy runs. It writes no variable and reads no value. + +- **The names it checks.** Every `envSecret()` in a service's input binding, at any depth; every secret in a module's input that no service reads; and every per-key param bound to `envParam()`. An `envParam()` in an input binding is not a platform variable: the deploy reads its text from the app configuration. +- **One list call per name.** The call is filtered by `projectId`, `class` and `key`, and walks every page. The class is `production` for the environment `production` and `preview` for every other. The environment sees the class row, and for a preview environment its own branch's row. At most eight calls run at once. A failed call fails the deploy. +- **The row the platform resolves.** The branch row wins when there is one, otherwise the class row. Preflight keeps that row's `updatedAt`. A change to a class row under a branch override changes nothing the service receives, so it triggers nothing. +- **A system row is refused.** A row the environment sees with `isManagedBySystem: true` fails the deploy, naming the variable. +- **Required or reported, per service.** For each service that reads a variable with no row, preflight serialises the input with every variable present and runs the service's schema. A failure there is an invalid input, reported as such and never blamed on a variable. Then it runs the schema with every variable that has no row left out. If that passes, those variables are optional for that service. If it fails, it leaves out each one alone, and the ones whose absence alone fails are required. When no single one fails alone, the error says the service needs at least one of them. A variable an array position reads is required without a check. This is the method [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md) gives the dev preflight, and both preflights call the same code. +- **A per-key param** is required when it has no default and is not optional. +- **A secret in a module's input that no service reads** has no reader to need it. It is reported, never required. +- **A required variable fails the deploy** with `DEPLOY.PREFLIGHT_FAILED`, before Alchemy runs: + + ```text + Deploy failed. 1 platform variable a service needs has no value: + - EMAIL_API_KEY (read by service "shop.email", field "apiKey") + + Set it on the platform, then deploy again: + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview --branch 'staging' + ``` + + For production the one command is `--role production`. For every other environment the preview-wide command comes first, so the next preview branch is not missing it too, and the branch command second. A variable that an array position or a per-key param makes required has a second line with the reason, and a line for a service that needs some of several variables ends with the schema's message. +- **The findings.** Every variable with no row that nothing requires is listed, with its readers (a service and field, a service and per-key param, or the module that names it and that no service reads it) and the commands that set it. `PreflightInput.report` takes a `PlatformVariableCheck`: the count of platform variables checked and this list, names and readers only, never a value. It carries platform-variable findings and that count, nothing else. The deploy prints the list right after the preflight step, under `Platform variables with no value:`. With `--json` it is a message whose data holds the list, and `onEvent` receives it as a `preflight-findings` event. The step's data stays `{ checked, unset }`, two counts. The CLI carries the list to the deploy's second process in core's own transport variable, `PRISMA_COMPOSER_PREFLIGHT_FINDINGS`, and lowering puts it into `DeploymentResult.findings`, so the deploy report repeats it at the end. An extension's own preflight payload never carries it. +- **Triggers.** Each service's deployment triggers carry `:updatedAt` for each platform variable its document or its per-key params point at, with `?` when the variable has no row. Changing a value on the platform and deploying again replaces exactly the deployments that read it. + +### `prisma dev` + +The local target plays the platform's part. + +- It fills each `envSecret()` variable from the app secrets and each per-key `envParam()` variable from the app configuration, under the variable's name. An app configuration value that is not text stops `prisma dev`, naming the variable, the service and the kind of value. +- The first process hands the second process the named secrets, the app secrets under the secrets keys of the graph's secret sources, in `PRISMA_COMPOSER_NAMED_SECRETS`, in memory. +- **It isolates.** A local service receives only the variables its own rows and `$platform` markers point at. This is a rule of the local target only: on Prisma Cloud every deployment receives every row of its class. +- A missing value follows ADR-0058: a required secret, a secret an array position reads, or a required per-key param stops `prisma dev` before any service starts, naming it and the service; an optional one is left out of the service's input, as the deploy's boot leaves it out. +- A changed value pauses rebuilds until `prisma dev` is restarted. After the restart, the deployment's `:updatedAt` trigger replaces only the readers' deployments. + +### The shortcuts + +- `envSecret('NAME')` is the one way to name a platform secret. Every value is text. +- `envParam('NAME')` in an input binding is a value from the app configuration at deploy, not a platform variable. It is deprecated: a field of the root's input replaces it. Its JSDoc carries `@deprecated`, so editors strike it through. +- `envParam('NAME')` in a service's per-key params is a platform variable, under every rule above. + +## Reasoning + +**An implicit copy moves a local value to a remote environment.** A deploy that copies a missing variable from the app secrets or the app configuration to the platform can carry a developer's test key to production, because `.env` is read for every environment. Copying on each deploy also writes one override row per preview branch. Until the operator can say explicitly which value goes to which environment, the operator sets each value on the platform, in the scope the operator chooses. + +**Preflight is the only protection against a missing value.** The platform promotes a deployment whose process exits at boot and reports it running. So a deploy that succeeded with a required variable unset would leave a crashing service in place. Preflight refuses the case it can see without a value: the service's own schema says the variable is needed. + +**The schema decides, as a black box.** [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) forbids reading a schema's internals. Leaving variables out and calling `validate` asks the schema exactly whether it can do without them, with no value and no introspection. + +**One call per name, not per scope.** The list filter by class and key already returns the class row and every branch row. Listing the whole class would return every branch's `COMPOSER_` rows on a preview project. + +**Upper-case names.** The platform accepts no other key, and Composer compares pointer names without case while a process environment on Linux compares them with case. With upper-case names a row and a name never differ by case, so boot finds the row preflight saw. + +**No leading underscore.** The platform accepts one; the rule is Composer's. Shells and tools use names that start with an underscore for their own variables: bash puts `_` in the environment of every command it runs, and the Java virtual machine reads `_JAVA_OPTIONS`. Keeping those names out of the app's names means an app's variable never meets one of them. + +**Empty means unset.** The platform cannot store an empty value. `prisma dev` treats empty as unset. Treating empty as unset at boot too makes the two agree. + +**The graph check runs after Load.** A rule that depends on more than one place in the graph cannot run in a constructor. Running it right after Load, in every command that runs Load, refuses a bad graph before preflight and before any dev service starts, without a deploy. + +## Consequences + +- A new environment needs its platform variables set first. For a new app, the first deploy creates the Project and Branch, then stops at preflight with the commands that set each required variable, including the project's id. +- Every new preview branch reads the preview-wide row. +- A deploy succeeds with an optional variable unset and lists it. Because the platform has no way to store "deliberately empty", the list also holds a variable whose absence is intended. +- A process that exits at boot for a reason preflight cannot see, such as a value that the schema refuses, is promoted. The guide names `prisma service logs ` as where the error is. +- Every service on Prisma Cloud sees every variable of its class. Composer records which service reads which variable in its graph and in each input document, so it can send that record if the platform ever takes one. +- Under `prisma dev` and on the platform, a service with an unset optional secret receives the same input: the field is absent. Dev's input document leaves the marker out; the deploy's document holds the marker, and boot leaves the key out. +- The deploy reads the app secrets only to redact the values under the graph's secrets keys wherever it prints. + +## Alternatives considered + +- **Copy local values on deploy.** Rejected: an implicit copy is how a local value reaches a remote environment by accident. +- **Succeed with every variable unset, and report.** Rejected: the platform promotes a service that exits at boot and reports it running, so the deploy would leave it crashing. Preflight refuses the case the schema can show without a value. +- **Two list calls per name, one per scope.** Rejected: one call filtered by class and key returns both. +- **The newest row's `updatedAt` as the trigger.** Rejected: a change to a class row under a branch override would redeploy a service whose value did not change. + +## Future work + +- **An explicit way to set platform variables through `deploy`.** The operator names the values and the environments they go to, and the deploy writes those rows. It needs its own design. It must not read local values implicitly. + +## Related + +- [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md): the secret. +- [ADR-0032](ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md): `envParam()` and per-key params. +- [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md): service input and the input document. +- [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md): how `prisma dev` reads secrets and decides required or optional. +- [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md): the app configuration and the app secrets. +- [deploying.md](../../guides/deploying.md) and [deploy-cli.md](../10-domains/deploy-cli.md). diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index 0e14af7bf..8872742d8 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -50,10 +50,10 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0026](ADR-0026-name-the-framework-prisma-compose.md) — The framework is **Prisma Composer**; "Prisma App" names only the artifact you build. Supersedes ADR-0014's framework, package, and CLI names: `@prisma/composer*` (incl. `compose-alchemy`), `prisma-composer`, `prisma-composer.config.ts`. - [ADR-0027](ADR-0027-two-packages-compose-and-compose-prisma-cloud.md) — Ship two **public** packages: `@prisma/composer` (core + CLI + agnostic subpaths) and `@prisma/composer-prisma-cloud` (target + first-party modules as entrypoints, cron first). Boundary = the user's one choice: where does it run. - [ADR-0028](ADR-0028-numbered-domains-and-layers-enforced-by-dependency-cruiser.md) — `packages/` organizes into numbered domains (0-framework, 1-prisma-cloud, 9-public) and layers; planes as config-mapped entrypoints; internals are `@internal/*`; only 9-public publishes; dependency-cruiser enforces. -- [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md) — A secret is a value the deployment reads from a platform variable by name, whose contents the framework never writes into deploy state, a generated file, a log, the deploy report or an error. A service types the field `secretString()`; the application binds it with `envSecret('NAME')`, a value whose real value is a secret pointer; the service reads a redacting `SecretString`. The input document carries `{ "$secret": NAME }`; preflight fills a missing platform variable from the app secrets with a write-only call. Composer redacts every secret it knows wherever it prints, at any length; under the CLI the deploy engine's own output is not redacted yet (ADR-0062). `secretSource()` takes a `secretsKey`, the app secrets key that holds the value. +- [ADR-0029](ADR-0029-a-secret-is-an-environment-sourced-value-the-framework-never-stores.md) — A secret is a value the deployment reads from a platform variable by name, whose contents the framework never writes into deploy state, a generated file, a log, the deploy report or an error. A service types the field `secretString()`; the application binds it with `envSecret('NAME')`, a value whose real value is a secret pointer; the service reads a redacting `SecretString`. The input document carries `{ "$platform": { name, secret: true } }`; the operator sets the variable on the platform (ADR-0064). Composer redacts every secret it knows wherever it prints, at any length; under the CLI the deploy engine's own output is not redacted yet (ADR-0062). `secretSource()` takes a `secretsKey`, the app secrets key that holds the value. - [ADR-0030](ADR-0030-rpc-callers-verified-with-an-auto-provisioned-service-key.md) — Every RPC binding carries a distinct, framework-minted **service key**: the client sends it, `serve()` rejects a caller without one (401). Auto-provisioned per edge at deploy, carried on the binding's own `COMPOSER_*` env-var rail; its value is minted and kept in the hosted deploy state (a capability token, deliberately not an ADR-0029 name-only secret). *(Proposed)* - [ADR-0031](ADR-0031-provisioned-param-values-are-a-need-resolved-through-a-target-registry.md) — A framework-minted param value is an opaque, branded **provisioning need** (not a named facet on the param): core forwards it and resolves its brand against the deploy target's `provisions` registry, failing loudly on a miss. The provisioner owns all mint/size/stability/rotation policy; core's surface stays one field. Resolves against the consumer's extension; cross-extension edges fail closed. *(Proposed)* -- [ADR-0032](ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md) — A service's per-key param can be bound at `provision()` to a literal or to a target-owned source such as `envParam('NAME')`, which reads `NAME` from the app configuration: a pointer row in params, filled by preflight and read at boot; the text in a service's input document; the real value in a module's input or a `resolve()`. Non-text fails the Load. `paramSource()` takes `configurationKey` and `redacted`. +- [ADR-0032](ADR-0032-params-bind-at-provision-env-sourcing-is-a-target-source.md) — A service's per-key param can be bound at `provision()` to a literal or to a target-owned source such as `envParam('NAME')`, which reads `NAME` from the app configuration: a pointer row in params, read at boot, where an unset variable gives the param its default, or leaves it out when optional; the text in a service's input document (deprecated in favour of a root input field); the real value in a module's input or a `resolve()`. Non-text fails the Load. `paramSource()` takes `configurationKey` and `redacted`. - [ADR-0033](ADR-0033-lowering-types-are-defined-by-their-readers.md) — Every value in the lowering pipeline is typed by the code that reads it, not the code that writes it, retiring the shared `LoweredNode` record: a descriptor types the values it passes between its own phases (`ServiceLowering`); the application hook's product reaches core as `unknown` and its own extension narrows it with a guard; a node's values for the nodes downstream are name-keyed `WiringOutputs`, resolved against the consumer's connection declaration. The lowering loop is the only party that knows which output feeds which input. Records the alchemy execution facts (a value passed between phases legitimately holds an unresolved `Output`), that the heterogeneous registry's type safety rests on the loop rather than the compiler, and that an unchecked claim is acceptable only when it is named, justified, and singular. - [ADR-0034](ADR-0034-deploy-state-lives-in-the-stage-branch.md) — Deploy state lives in a framework-owned `prisma-composer-state` database in the stage's own Branch (production: the implicit default Branch). State has the environment's lifetime: platform-side Branch/Project deletion cleans it up with no framework involvement; the CLI deletes it last-among-members on destroy. Supersedes ADR-0009's workspace-level store. *(Superseded in part by ADR-0045: the Branch scoping and lifetime stand; the per-stage database is replaced by the platform state API.)* - [ADR-0035](ADR-0035-public-entrypoints-live-in-src-exports.md) — Public entrypoints live in `src/exports/` (one file per subpath; internals stay at the `src/` root); `@internal/tsdown-config` generates `package.json#exports` from object-named entries where safe, with two deliberate exceptions kept hand-maintained (the multi-pass `cron`/`storage`/`streams`, and the two published packages). Completes ADR-0028. @@ -63,7 +63,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0039](ADR-0039-a-compute-services-own-origin-is-a-target-resolved-property.md) — A compute service's own platform-assigned origin is a target-resolved property, read as the service's `origin()` — never a declared param, never operator config, never in `config()`. It rides ADR-0031's reserved provider-param channel as the first *service-derived* entry (`valueForService(provisioned, address)`, written for every compute service, exposing or not), sourced from the provisioned App's `appEndpointDomain` (ADR-0048) — made trustworthy pre-promote by the upstream PRO-200 fix. `envParam(…)` remains correct for operator-known origins (custom domains); narrows ADR-0032's `appOrigin` example accordingly. - [ADR-0040](ADR-0040-the-orm-binding-carries-the-url-and-a-lazy-client.md) — `postgres(contract)`'s dependency binding is `{ url, client }`: the raw connection string plus the typed client, constructed lazily and memoized on first `client` access — `hydrate` builds nothing. The contract remains the compatibility interface (hash check and deploy-time migration unchanged, ADR-0022); the binding becomes a strict superset of plain `rawPostgres()`'s `{ url }`, so an app that owns its database client still gets framework-run migrations. Contract validation cost and failure move from `load()` (where one bad input poisoned every input, unattributed) to the first `client` access. Cross-kind satisfaction (`'postgres'` satisfying `'postgres'`) rejected in its favor. *(Extended by ADR-0055: the binding also carries `connection` and `clientWith(options)`.)* - [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres the Postgres emulator hosts one `@prisma/dev` server per database instance. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev`; logs are the `log` operation, not a command. Amended 2026-10-23: the entry is the `composer` section's `entry` or `--entry`, not a positional. Amended by ADR-0052: emulator instances are keyed by emulator-generated ids, not the app name, Composer maps each app's nodes to them per directory, and `prisma dev` stops the app's database servers when it ends.)* *(Amended by ADR-0056: a service runs the dev server its build adapter declares in place of its artifact, as a command deployment on the Compute emulator.)* *(Amended by ADR-0062: local dev runs as a named environment, `dev` by default.)* -- [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares all of its input, plain values and secrets together, as one Standard Schema (`input`), read back through `service.input()`. The binding at `provision()` is typed from the schema's input type, with the same type a module's binding has, so a wrong literal or a source in the wrong kind of field fails to compile. Secret fields are `secretString()`; a field that may receive a value known only at boot is wrapped in `field()`. The deploy checks the binding with the schema, each value known only at boot in place as a pointer, and writes one self-describing input document per service with `$secret` and `$generated` markers; boot reads it, boxes the secrets and checks it again. Composer never introspects the schema. An unset `envParam()` leaves its key out, and the schema decides whether that is allowed. *(Its current text includes ADR-0061's extension to modules.)* +- [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares all of its input, plain values and secrets together, as one Standard Schema (`input`), read back through `service.input()`. The binding at `provision()` is typed from the schema's input type, with the same type a module's binding has, so a wrong literal or a source in the wrong kind of field fails to compile. Secret fields are `secretString()`; a field that may receive a value known only at boot is wrapped in `field()`. The deploy checks the binding with the schema, each value known only at boot in place as a pointer, and writes one self-describing input document per service with `$platform` and `$generated` markers; boot leaves out a field whose platform variable is unset or empty, boxes the secrets and checks it again. Composer never introspects the schema. An unset `envParam()` leaves its key out, and the schema decides whether that is allowed. *(Its current text includes ADR-0061's extension to modules.)* - [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md) — `@prisma/composer/control` is the programmatic deploy surface: typed `deploy`/`destroy`/`dev`/`log` operations (structured inputs/results, no argv/console/exit) implemented in `@internal/cli`'s `src/operations/` and re-exported per ADR-0035; the CLI is a thin renderer over them. The entry's static graph stays import-light — each operation lazily imports its executor, so importing the subpath executes nothing, and a tree that cannot load the deploy stack surfaces as a structured `pipeline` failure — and `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` carries the deploy result across the process boundary: the alchemy child's report hook writes a serializable `DeploymentSummary` to the named file, the operation reads it back best-effort (absent/malformed = undefined summary, never a failure). Distinct from an extension's ADR-0017 `/control` entry. *(Amended by ADR-0049: each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired.)* *(Amended by ADR-0050: the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`; `destroy` and `log` are called from scripts.)* *(Amended by ADR-0053: `devUrl` and `devStop` join the operations.)* *(Amended by ADR-0059: `deploy` and `destroy` take `environment`, which `destroy` requires.)* *(Amended 2026-10-28: `devUrl` and `devStop` act on the app and environment in their input; `dev` and `log` take `environment`.)* - [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes from a closed registry, structured at origin with why/fix splits, no catch-all codes; bugs carry no code (exit 1); recognition is structural; operation results ride the `Result` `ok` discriminator; expected failures exit 2, with the deploy engine's status passed through. The one change between origin and consumer is that the operation redacts known secrets and drops the cause. - [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md) — Deploy state lives behind the platform state API (the Management API implements Alchemy's stock `HttpStateApi` wire contract per Branch; composer's state layer is Alchemy's stock HTTP client), and deploys hold a server-side per-`(stack, stage)` lease (TTL 60s, heartbeated, released on exit; contention fails fast naming the holder; state operations without a live lease fail 409). Supersedes ADR-0010 (lock → lease) and the storage half of ADR-0034 (Branch scoping and lifetime stand; the visible per-stage database is gone); closes ADR-0012 as obsolete. No migration: legacy stages are refused until destroyed or deleted. @@ -79,9 +79,10 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0055](ADR-0055-the-postgres-binding-builds-clients-with-the-apps-options.md) — The `postgres(contract)` binding gains `clientWith(options)`, which builds a typed Prisma ORM client with the runtime's own options (less `poolOptions`), such as pgvector's runtime extension or `middleware`, and `connection`, the pool every client shares, for an app that builds its client itself. The dependency declaration takes no options: they are how the service consumes the database, not what it requires, and passing the runtime's option type through means a new option needs no Composer change. `client` is unchanged. Composer never discovers an extension package from the contract's pack ids; a missing one still fails when the client is built, with the runtime's error. Rejected: options on the declaration, extensions on the contract. Amends ADR-0022, extends ADR-0040. - [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) — Under `prisma dev`, a service runs the dev server its build adapter declares, in place of its built artifact: `nextjs()` declares `next dev`; `node()` declares none unless the author writes `dev: [...]` on it. No flag. The build descriptor gains `serve(input) → DevServer | undefined` (`cwd`, `argv({ port, host })`, `display`), called before assembly; such a service is not assembled and lowers without its `Deployment` (`LowerOptions.unassembled` + the service descriptor's `reserve`), so its rows, instance, port and dependents' URLs stay. The Compute emulator runs the dev server as a *command deployment* (`{ command: { cwd, argv }, env, listens }`) on the service's instance, with the deployment's materialized env re-stashed address-free by `stashRows`, the pure function `run()` now uses, plus `PORT`/`HOST`, over the shell's env; readiness, logs, crash supervision, restart on env change, `--detach`, `dev stop` and session resume are unchanged and the dev session owns no processes. The session streams dev servers' logs inline; `prisma dev` also watches the entry and every service's authoring module; `devUrl` gains `runs`. New codes `DEV.DEV_SERVER_UNAVAILABLE`, `DEV.DEV_SERVER_UNSUPPORTED`. Amends ADR-0041 and ADR-0053; extends ADR-0005. - [ADR-0057](ADR-0057-a-nodes-address-is-its-full-path-from-the-app-name.md) — A node's address is the path of provision IDs starting at the root, whose address is the deployed app name (letters, digits, `-` and `_`; `shop`, `shop.catalog`, `shop.auth.api`), and it is the node's only identity. It is the `logicalId` of the node's platform row and topology node; Alchemy resource IDs are `
` for the main resource and `
.` for other pieces, which cannot collide because a module has children and no resources and a service or resource has resources and no children; `COMPOSER_` env keys are built from the full address with `-` turned into `_`; `serviceOrigin()` takes the full address. Every platform name Composer writes is the address, a label; the upgrade recreates Composer's Connections and bucket access keys with new credentials. Existing stages migrate on their next deploy: every resource declares `renamedFrom` its previous ID, and generated values, service keys and stream keys are renamed, not recreated; local dev's record is rewritten on first run. An older Composer deploying a migrated stage is not protected against. Rejected: unqualified child addresses, a second shorter "path" form, `-` suffixes on resource IDs, relative `serviceOrigin()`. Supersedes ADR-0051, carrying forward its identity rules. -- [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md) — Under `prisma dev` a secret comes from the app secrets and an env-sourced param from the app configuration, read in the first process at start and again at each rebuild; Composer never invents a value or reuses one. A required secret that is missing stops dev; an optional one arrives absent, decided by validation. The second process reads the app configuration file, deleted after each converge, and receives the named secrets in one environment variable, never a file. Fingerprints of values on the preflight transport redeploy exactly the services whose value changed. +- [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-app-secrets-and-never-invents-them.md) — Under `prisma dev` a secret comes from the app secrets and an env-sourced param from the app configuration, read in the first process at start and again at each rebuild; Composer never invents a value or reuses one. A required secret that is missing stops dev; an optional one arrives absent, decided by validation. The second process reads the app configuration file, deleted after each converge, and receives the named secrets in one environment variable, never a file. An HMAC fingerprint of each value, passed to the second process, redeploys exactly the services whose value changed. A per-key `envParam()` param stops dev only when it is required. Each local service receives only its own variables; that isolation is a local-target rule. - [ADR-0059](ADR-0059-a-deployment-environment-is-called-an-environment.md) — A place the app runs is an **environment**: a target-neutral concept of Composer's core, used on every public surface (`--environment`, `destroy`'s target, guides, errors, skill), replacing "stage". Every environment has a name and none is special: `prisma deploy` defaults to `production`, `prisma dev` to `dev`. On Prisma Cloud, `production` is the Project's default Branch and any other environment is the Branch of that `gitName`; `dev` has no Branch. "Branch" is Prisma Cloud's term and its git-shaped workflow, and appears only where Composer reports what the target did. - [ADR-0060](ADR-0060-public-traffic-reaches-a-service-through-a-public-url.md) — Traffic from the internet reaches a service only through a **public URL** the author declares: `publicUrl({ id, url, target })`, a driver wired to the HTTP connection a service exposes, as `cron` is wired to what it calls. `url` is one of two explicit strategies, a named hostname or the prisma.build URL the platform gives every service, usually computed per environment with `resolve()`. One hostname per public URL. A reusable module decides what can be public by exposing connections; the app decides what is. Services stay publicly reachable underneath until the platform can address them privately. On Prisma Cloud a named hostname lowers to the platform's RoutingUrl record, covered by a domain ownership claim the workspace has verified; a prisma.build URL lowers to nothing, because every service keeps the platform's own hostname. *(Amended 2026-10-29: the public URL node is a resource by kind and a driver by role, with a slot wired at construction and `props`; see the note at its top.)* *(Amended 2026-11-02: `publicUrl()` takes its identity as `id`, not `name`.)* - [ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md) — A module declares its input as one Standard Schema, `input`, exactly as a compute service does, and whoever provisions it binds that input with a binding typed from the schema. The body receives `input` as a dictionary with one value per field; it passes fields on by name, or lists a plain field in `resolve()`, and never reads their data, so the graph cannot branch on its input. The dictionary itself is not a value: passing it on whole fails to compile and fails at Load, for every module. A field that may receive a value known only at boot is wrapped in `field()`; a secret is a `secretString()` field. The module `params`/`paramNeed()` and `secrets`/`secret()` slots are removed. The root may declare `input`, bound to the app configuration. -- [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — The graph's structure is the same whatever inputs Composer is given; only values differ per environment. A module body holds values (`Value`), opaque stand-ins for data known later: deploy-time values, whose data the deploy has by the end of Load, and runtime values (`RuntimeValue`, which is `Value>`), whose data only the platform holds. `resolve(dependencies, fn)` computes a value from deploy-time values only; `fn` never sees a pointer. Config adapters produce the app configuration, which is the root's input and what `envParam()` reads; secrets adapters produce the app secrets, which fill missing `envSecret()` platform variables. Both lists default to `[environmentVariables()]`; `dotenv()` reads `.env` files through `dotenv-flow`. Only the first process runs adapters, once per deploy; the second reads the app configuration file and refuses a graph whose node addresses differ. Secrets stay out through pointers and redaction. Every environment has a name and none is special. +- [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — The graph's structure is the same whatever inputs Composer is given; only values differ per environment. A module body holds values (`Value`), opaque stand-ins for data known later: deploy-time values, whose data the deploy has by the end of Load, and runtime values (`RuntimeValue`, which is `Value>`), whose data only the platform holds. `resolve(dependencies, fn)` computes a value from deploy-time values only; `fn` never sees a pointer. Config adapters produce the app configuration, which is the root's input and what `envParam()` reads; secrets adapters produce the app secrets, which `prisma dev` gives its services and which Composer redacts; the deploy does not copy them to platform variables, which the operator sets on the platform. Both lists default to `[environmentVariables()]`; `dotenv()` reads `.env` files through `dotenv-flow`. Only the first process runs adapters, once per deploy; the second reads the app configuration file and refuses a graph whose node addresses differ. Secrets stay out through pointers and redaction. Every environment has a name and none is special. - [ADR-0063](ADR-0063-a-node-depends-on-an-interface-or-exposes-one.md) — The two ends of a connection are named by what the node does with them: a node **depends on** an interface (`deps`) or **exposes** one (`expose`); an interface is a `Contract` in code; Input and Output are retired as port words, because the dataflow and ports-and-adapters traditions give the same two ends opposite names; the module body's `inputs` becomes `deps`; the topology's `in`/`out` stay, defined as the dependency end and the exposed end. +- [ADR-0064](ADR-0064-the-deploy-does-not-copy-local-values-to-platform-variables.md) — The deploy does not copy local values (by default `.env` and the shell) to platform variables, because an implicit copy can push a local value into a remote environment. For now the operator sets each variable on the platform with `prisma project env add`; an explicit way to set variables through `deploy` is expected later. Before Alchemy runs, the deploy checks that every variable a service needs has a value, and fails with the commands that set the missing ones. Variable names match `^[A-Z][A-Z0-9_]*$`, and a few names are reserved for Composer and the platform. diff --git a/docs/guides/building-an-app.md b/docs/guides/building-an-app.md index 4f39a9a71..d01bb6adb 100644 --- a/docs/guides/building-an-app.md +++ b/docs/guides/building-an-app.md @@ -675,13 +675,17 @@ keeps the secret box when it transforms, so it can do both. **`envParam('NAME')`** is for plain values the code *can't* know. An app origin is the canonical case: it's different on production and on every preview environment. The deploy reads the variable from the app configuration: by default the shell that runs `prisma deploy` or `prisma dev`, or the `config` adapters you list ([Values from outside the code](#values-from-outside-the-code)), and writes its value into the service's input. So set it in that shell for every deploy; in CI, export it per environment. An unset or empty variable leaves the key out, and the schema decides ([below](#absence-is-the-schemas-call)). The value is text, so bind `envParam` to text fields. Binding it to a field of a narrower type, such as `'resend' | 'smtp' | 'none'`, fails to compile, because the variable may hold any text: declare that field in your root's `input` with the same type instead (for `email()`, build it from the exported `emailDeliveryModes`), and the root's schema checks the variable before anything is created. To change the value, deploy again with the new value. [`examples/env-param`](../../examples/env-param/) is the minimal working version, including a smoke script that proves the per-environment split. -**`envSecret('NAME')`** is for credentials, with one rule: **the value never enters framework config**. Each environment keeps its own copy of the platform variable, and that copy is the store: at deploy, preflight checks that the environment has the variable, and copies a missing one up from the app secrets: by default the deploying shell, so CI (or you) exports `STRIPE_SECRET_KEY` for a first deploy. The framework carries only the variable's *name*; the platform injects the value straight into the running instance. What your code gets is a `SecretString` box that redacts on logging, rendering, and JSON-serialization — leaking it takes a deliberate `.expose()`: +**`envParam()` in an input binding is deprecated.** Declare the value as a field of your root's `input` instead and pass the field on by name ([Values from outside the code](#values-from-outside-the-code)): the root's schema then checks it before anything is created. It still works. Only `envParam()` in an input binding is deprecated; `envParam()` in a service's per-key params is not. Editors strike through every `envParam()` call, including in per-key params, because one function serves both uses. The deprecation applies only to an input binding. + +**`envSecret('NAME')`** is for credentials, with one rule: **the value never enters framework config**. The value lives in a platform variable that you set on the platform, for production, for every preview environment, or for one: `prisma project env add 'STRIPE_SECRET_KEY=' --project '' --role preview`. The deploy does not copy it from `.env` or your shell. At deploy, preflight checks that the environment has the variable, and fails, with the command to run, when a service's schema requires a variable that has no value ([Deploying § Platform variables](deploying.md#platform-variables)). The framework carries only the variable's *name*; the platform injects the value straight into the running instance. Under `prisma dev` the value comes from the `secrets` adapters instead. What your code gets is a `SecretString` box that redacts on logging, rendering, and JSON-serialization — leaking it takes a deliberate `.expose()`: ```ts const input = service.input(); if (input.stripeEnabled) stripe(input.stripeSecretKey.expose()); ``` +`envSecret()` and `envParam()` take an upper-case name: an upper-case letter followed by upper-case letters, digits and underscores, such as `STRIPE_SECRET_KEY`. Any name starting with `COMPOSER_`, and `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` and `HOME`, are reserved. A name has one meaning: `envParam('X')` and `envSecret('X')`, or `envParam('X')` and a field `X` of your root's input, cannot both appear in one app. Each of these fails before anything is created, naming the rule. + A reusable Module takes a secret as a `secretString()` field of its `input` and passes it on without ever learning the platform name ([above](#reusable-modules)), which is what lets @@ -689,7 +693,7 @@ a Module require credentials without dictating your naming. ### Absence is the schema's call -An env-bound field whose variable is unset (or empty) in the app configuration resolves to *key omitted*; whether that is legal is the schema's call — an optional field, a union arm, or a validation error that fails the deploy. A credential for an off-by-default feature is an ordinary optional `SecretString` field, not a framework flag. Because an omitted key can also be a typo'd variable name, the deploy report prints every key that resolved absent. +An env-bound field whose variable is unset (or empty) resolves to *key omitted*: an `envParam()` field at deploy, from the app configuration, and an `envSecret()` field at boot, from the platform. The schema decides whether that is allowed: an optional field, a union member, or a validation error that fails the deploy. A credential for an off-by-default feature is an ordinary optional `SecretString` field, not a framework flag. Because an omitted key can also be a mistyped variable name, the deploy report prints every key that resolved absent. ### What travels @@ -699,13 +703,10 @@ pointer naming the platform variable that holds the value — never the value itself: ```json -{ "stripeEnabled": true, "stripeSecretKey": { "$secret": "STRIPE_SECRET_KEY" } } +{ "stripeEnabled": true, "stripeSecretKey": { "$platform": { "name": "STRIPE_SECRET_KEY", "secret": true } } } ``` -At boot the framework swaps each pointer for a redacting box over the named -variable, validates against the schema again, and `service.input()` returns -the typed object. The document is secret-free by construction, which is why -the deploy report can print it verbatim. +At boot the framework swaps each marker for a redacting box over the named variable, or leaves the field out when the variable is unset or empty, validates against the schema again, and `service.input()` returns the typed object. When the schema fails and a field was left out, the error ends with the names of the variables that had no value. A marker inside an array cannot be left out, so an unset variable there fails boot, naming it. The document holds no secret values, which is why the deploy report can print it, with each marker shown as `$platform NAME`. ## Values that differ per environment @@ -904,8 +905,7 @@ the shell. `envParam('NAME')` reads `NAME` from the app configuration. -The `secrets` adapters fill a missing `envSecret()` platform variable at -deploy, and give `prisma dev` its secrets. +The `secrets` adapters give `prisma dev` its secrets. A deploy reads them only to know which values to redact. The operator sets a platform variable on the platform; the deploy does not copy it from the `secrets` adapters ([Deploying § Platform variables](deploying.md#platform-variables)). A root's input takes no secret, so bind `envSecret()` inside the body, as `stripeKey` is above. diff --git a/docs/guides/deploying.md b/docs/guides/deploying.md index 1a5f794c6..dcb5d4b81 100644 --- a/docs/guides/deploying.md +++ b/docs/guides/deploying.md @@ -244,14 +244,7 @@ dependencies are traced and copied into the deploy artifact (see Planning and applying run in one deploy engine process, so they are one step. The last line is the whole deploy, from start to finish. -With `--json`, each step is a -`step-started` and a `step-finished` line on stdout, followed by an -`endpoint` line per public URL and the `result` line. A `step-finished` line -carries `data.durationMs`. An assemble step's `data` also holds whatever the -service's build adapter reports, and the env var check's `data` holds -whatever the deploy target reports. Those extra fields vary by adapter and -target, so read them as information, not as a stable format. The `result` -line has the deploy's total `durationMs` beside `summary`. +With `--json`, each step is a `step-started` and a `step-finished` line on stdout, followed by an `endpoint` line per public URL and the `result` line. A `step-finished` line carries `data.durationMs`. An assemble step's `data` also holds whatever the service's build adapter reports, and the `check environment variables` step's `data` holds `checked` and `unset`, how many platform variables it checked and how many have no value. When some have no value and no service needs them, a `message` line follows the step, whose `data.unset` lists each one with its readers and commands ([Platform variables](#platform-variables)). The build adapter's fields vary by adapter, so read them as information, not as a stable format. The `result` line has the deploy's total `durationMs` beside `summary`. A deploy ends by printing your app's own topology — the names you authored, what each one became on the platform, and the public URLs: @@ -286,6 +279,63 @@ A node that deployed but published nothing reportable is still listed, marked about your app. It's gone, replaced by the tree above. Nothing you can configure printed it and nothing depended on it. +## Platform variables + +A platform variable is an environment variable on Prisma Cloud that a service reads at boot: the one an `envSecret('NAME')` names, or a service param bound to `envParam('NAME')`. You set it on the platform. The deploy does not copy a value from `.env` or your shell to the platform, because that could push a local value into a remote environment by accident, and today it creates, changes and deletes no platform variable. + +Set a variable with the `prisma` CLI, at one of three scopes: + +```sh +# production +prisma project env add 'EMAIL_API_KEY=' --project '' --role production +# every preview environment +prisma project env add 'EMAIL_API_KEY=' --project '' --role preview +# one preview environment only, overriding the preview-wide value +prisma project env add 'EMAIL_API_KEY=' --project '' --role preview --branch 'staging' +``` + +`update`, `list` and `delete` take the same flags. A deployment receives every variable of its class with its own branch's overrides on top, so a preview-wide value reaches every new preview environment. + +The platform copies the variables into a deployment when it creates it. A value you change reaches a service at its next deploy, and that deploy replaces exactly the services that read the variable. + +### What the deploy checks + +The `check environment variables` step looks up every platform variable the app reads, before any service or database is created. It reads whether a value exists, never the value. + +- **A variable a service needs that has no value fails the deploy.** Whether a service needs it is decided by the service's input schema: the deploy leaves the variable out and validates the input. A required field fails; an optional one does not. A param with no default that is not optional is needed too. + + ```text + Deploy failed. 1 platform variable a service needs has no value: + - EMAIL_API_KEY (read by service "shop.email", field "apiKey") + + Set it on the platform, then deploy again: + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview + - prisma project env add 'EMAIL_API_KEY=' --project 'proj_…' --role preview --branch 'staging' + ``` + + For `production` the command uses `--role production`. For any other environment the preview-wide command comes first, so that the next preview environment has the value too. +- **A variable nothing needs that has no value is listed, and the deploy goes on.** The list prints right after the step, and again at the end of the deploy output: + + ```text + ✔ check environment variables (1.2s) + Platform variables with no value: + - STRIPE_WEBHOOK_SECRET (read by service "shop.billing", field "webhookSecret") + prisma project env add 'STRIPE_WEBHOOK_SECRET=' --project 'proj_…' --role production + ``` + + The service starts without that field, and its schema decides what that means. A secret in a module's input that no service reads is listed the same way. +- **A variable Prisma Cloud manages itself** (`isManagedBySystem`) fails the deploy, naming it. Rename it in your app. + +A new app's first deploy creates its project, then stops at this step if a service needs a variable, with the commands and the new project id. Set the variables and deploy again. + +At boot, an unset or empty variable leaves its field out of the service's input, and the service's schema decides. A param with a default takes the default. A service whose schema refuses the input fails at boot, and the error ends with the names of the variables that had no value. **Prisma Cloud still reports a service that fails at boot as running, and the deploy promotes it**, so the deploy succeeds; the error is in `prisma service logs --project `. + +Every deployment on Prisma Cloud receives every variable of its class, including the ones another service reads. Under `prisma dev` each service receives only its own. + +### Names + +A name is an upper-case letter followed by upper-case letters, digits and underscores, such as `STRIPE_SECRET_KEY`. These names are reserved: any name starting with `COMPOSER_`, `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` and `HOME`. A name has one meaning in an app: `envParam('X')` and a field `X` of your root's input cannot both read it, and neither can `envParam('X')` and `envSecret('X')`. Each of these fails before anything is created, naming the rule. + ## Listing what is deployed `prisma deploy list` shows every node the app has deployed in an environment, @@ -382,9 +432,7 @@ example `bun destroy-staging.ts`, or `node destroy-staging.ts` on a supported No The three teardown shapes differ in what happens to state. Destroying any environment other than production removes its resources, then deletes its Branch — and the Branch takes the environment's deploy state with it. Destroying **production** removes the resources and empties production's deploy state as it goes, but the production Branch survives, so an emptied state scope remains until the Project itself is removed. Deleting the **Project** (below, or from the Console) removes every Branch and all state in one stroke. Destroy never creates: tearing down an environment that was never deployed fails with "nothing deployed" rather than provisioning one first. -Destroying production also removes the app's Project once nothing is left in -it, so hand-run stacks don't pile up as empty Projects in your workspace. If -the Project still holds another environment's resources, it's left in place. +Destroying production also removes the app's Project, so hand-run stacks don't pile up as empty Projects in your workspace. If the Project still holds another environment's resources, it's left in place. Removing the Project deletes every platform variable set on it, so set them again after the next deploy creates the Project. ## CI @@ -394,7 +442,7 @@ pattern: on push, run `prisma deploy --environment "pr-$PR_NUMBER"`; on close, run a script like the one under [Removing an app or an environment](#removing-an-app-or-an-environment) that passes that name, `environment: 'pr-42'` for PR 42. -One extra: if your app binds `envSecret` (see [Building an app § Service input](building-an-app.md#service-input)), each environment keeps its own copy of those platform variables, and the platform copy is the store; the app secrets only seed it. A fresh environment (a new `pr-42`) has none of them yet, so CI must export the values (e.g. `AUTH_SIGNING_SECRET`) alongside the two credentials; preflight copies missing ones up on that first deploy. A name absent from both the platform and the app secrets fails the deploy early, naming the missing variable. An `envParam` input is different: the deploy reads it from the app configuration on every deploy, so CI exports it, or commits it in `.env.` and lists `dotenv()`, every time. +One extra: if your app binds `envSecret` (see [Building an app § Service input](building-an-app.md#service-input)), its platform variables live on the platform, and CI does not supply them. Set each one once with `--role preview`, and every new `pr-42` environment reads it ([Platform variables](#platform-variables)). A variable a service needs that has no value fails the deploy, naming it. An `envParam` input is different: the deploy reads it from the app configuration on every deploy, so CI exports it, or commits it in `.env.` and lists `dotenv()`, every time. A root's input and an `envParam()` can come from a committed `.env.` file per environment ([Values from outside the code](building-an-app.md#values-from-outside-the-code)). Keep secrets out of committed files: put them in CI's secret store and keep `environmentVariables()` in `secrets`. @@ -729,14 +777,7 @@ What to know before embedding it: not something to build on; branch on `message`/`cause` for anything durable. Importing the module executes nothing until you call an operation. - The deploy engine's own output passes through Composer on its way to your terminal, with every secret Composer knows replaced by `[redacted]`: the value under each name an `envSecret()` names, in your `config` and `secrets` lists, however short. Other values in those lists, such as the shell's, are printed as they are, apart from values that look like credentials, such as a long value under a name containing `TOKEN` or `KEY`. So the engine shows plain progress lines, without colour. -- **`onEvent` reports each step.** Pass `onEvent` to `deploy` to receive a - `step-started` and a `step-finished` event per step (`prepare`, `assemble`, - one `assemble-service` per service, `connect`, `preflight`, `apply`, - `record`), as they happen. A finished step has `outcome`, `durationMs` and - the `data` described in [What a deploy prints](#what-a-deploy-prints). When - a step fails, it and the steps around it finish as `failed`; `record` still - runs. An `onEvent` that throws does not fail the deploy. A successful - result's `durationMs` is the whole deploy's total. +- **`onEvent` reports each step.** Pass `onEvent` to `deploy` to receive a `step-started` and a `step-finished` event per step (`prepare`, `assemble`, one `assemble-service` per service, `connect`, `preflight`, `apply`, `record`), as they happen. A finished step has `outcome`, `durationMs` and the `data` described in [What a deploy prints](#what-a-deploy-prints). Right after `preflight`, a `preflight-findings` event lists the platform variables with no value that no service needs, when there are any. When a step fails, it and the steps around it finish as `failed`; `record` still runs. An `onEvent` that throws does not fail the deploy. A successful result's `durationMs` is the whole deploy's total. - **`summary` is best-effort.** It rides a result file the deploy engine's child process writes; a deploy that converged without writing one still succeeds, with `summary: undefined`. diff --git a/docs/guides/running-locally.md b/docs/guides/running-locally.md index c138ae2c8..4c98bc9bd 100644 --- a/docs/guides/running-locally.md +++ b/docs/guides/running-locally.md @@ -422,13 +422,13 @@ token, workspace, or network is involved. Five consequences worth knowing: If a secret your service's input schema requires is unset or empty, `prisma dev` stops before it starts anything and names the variable and the service. If the schema marks the secret optional, the service starts and sees that field as absent. Change a value and run `prisma dev` again, without `--fresh`, and the services that read it restart with the new value. - **`.env` and `.env.local` are read by every environment, including `prisma deploy` to `production`.** Deploy uploads any secret the target environment does not have yet, so a local-only secret in those files becomes the hosted value. Put local-only values in `.env.dev` or `.env.dev.local`. + **`.env` and `.env.local` are read by every environment, including `prisma deploy` to `production`.** The deploy does not copy a secret to the platform, but a plain value in those files becomes the deployed root input or `envParam()` value. Put local-only values in `.env.dev` or `.env.dev.local`. A `.env` that only your framework loads (Next.js, for example) does not count, because Composer reads each secret before the dev server starts (see [Dev servers](#dev-servers)). `prisma dev` treats as secrets only the values under the names an `envSecret()` in the graph reads, and redacts those values in what it prints. The other values in `.env` and the shell are not secrets to it, and print as they are, apart from values that look like credentials, such as a long value under a name containing `TOKEN` or `KEY`. - Each service gets only the secrets and env-sourced params its own input and params point at. Code that reads `process.env.X` directly, where `X` is bound by another service, gets nothing. + Each service gets only the secrets and env-sourced params its own input and params point at. Code that reads `process.env.X` directly, where `X` is bound by another service, gets nothing under `prisma dev`. On Prisma Cloud every service receives every platform variable of its class, so do not rely on this isolation there. - Changing a value an adapter reads while `prisma dev` runs pauses rebuilds at the next rebuild, with a message; restart `prisma dev` to use the new values. - **The emulators outlive a session.** They're shared, machine-wide daemons, so your data survives `Ctrl-C` and even a reboot until you `--fresh`. That's diff --git a/examples/auth/.env.example b/examples/auth/.env.example index 146c00391..16f6426e4 100644 --- a/examples/auth/.env.example +++ b/examples/auth/.env.example @@ -1,5 +1,6 @@ # Local values. Copy this file to .env; dotenv() in prisma.config.ts reads it, and the shell wins over it. -# Also read by `prisma deploy` from this directory, which uploads any secret the target environment does not have yet and writes these params into the deployed input: deploying with this .env loaded gives the deployed environment a mode that sends no email. +# `prisma deploy` run from this directory also reads it and writes these values into the deployed service's input. Deploying with this .env loaded gives the deployed environment a mode that sends no email. +# The deploy does not copy a secret from this file to the platform. Set EMAIL_DELIVERY_CREDENTIAL there with prisma project env add. # Delivery mode none sends no email, so the credential is never used. EMAIL_DELIVERY_MODE=none EMAIL_FROM=dev@example.com diff --git a/examples/storefront-auth/.env.example b/examples/storefront-auth/.env.example index c548d1c8b..4c7e35e6b 100644 --- a/examples/storefront-auth/.env.example +++ b/examples/storefront-auth/.env.example @@ -1,4 +1,4 @@ # Local values for `prisma dev`. Copy this file to .env; prisma.config.ts loads it. -# Also read by `prisma deploy` from this directory, which uploads any secret the target environment does not have yet, so choose a value of your own and keep it out of deployed environments. +# The deploy never copies a secret from this file to the platform: set AUTH_SIGNING_SECRET for a deployed environment with prisma project env add. # prisma dev stops with an error naming this variable until you set it. AUTH_SIGNING_SECRET= diff --git a/examples/storefront-auth/module.ts b/examples/storefront-auth/module.ts index 1f5b048be..6cc76931e 100644 --- a/examples/storefront-auth/module.ts +++ b/examples/storefront-auth/module.ts @@ -11,8 +11,6 @@ import storefrontService from '@storefront-auth/storefront'; * that contract. */ export default appRoot('storefront-auth', ({ provision }) => { - // The root binds the auth module's signingKey to the platform variable - // AUTH_SIGNING_SECRET; preflight fills it from the app secrets when missing. const auth = provision(authModule, { input: { signingKey: envSecret('AUTH_SIGNING_SECRET') }, }); diff --git a/examples/storefront-auth/modules/auth/package.json b/examples/storefront-auth/modules/auth/package.json index 432f7724e..53a9123b9 100644 --- a/examples/storefront-auth/modules/auth/package.json +++ b/examples/storefront-auth/modules/auth/package.json @@ -6,6 +6,7 @@ "exports": { ".": "./src/module.ts", "./contract": "./src/contract.ts", + "./key-check": "./src/key-check.ts", "./fake": "./testing/fake.ts" }, "scripts": { diff --git a/examples/storefront-auth/modules/auth/src/contract.ts b/examples/storefront-auth/modules/auth/src/contract.ts index 38121e0b6..bbc0e8a69 100644 --- a/examples/storefront-auth/modules/auth/src/contract.ts +++ b/examples/storefront-auth/modules/auth/src/contract.ts @@ -8,7 +8,5 @@ import { type } from 'arktype'; export const authContract = contract({ verify: rpc({ input: type({ token: 'string' }), output: type({ ok: 'boolean' }) }), - // Non-leaking proof that auth received its injected `AUTH_SIGNING_SECRET` - // (ADR-0029): returns ONLY a boolean, never the secret value. - secretCheck: rpc({ input: type({}), output: type({ ok: 'boolean' }) }), + keyCheck: rpc({ input: type({}), output: type({ value: 'string' }) }), }); diff --git a/examples/storefront-auth/modules/auth/src/key-check.ts b/examples/storefront-auth/modules/auth/src/key-check.ts new file mode 100644 index 000000000..3dcc7a517 --- /dev/null +++ b/examples/storefront-auth/modules/auth/src/key-check.ts @@ -0,0 +1,4 @@ +import { createHmac } from 'node:crypto'; + +export const keyCheckValue = (signingKey: string): string => + createHmac('sha256', signingKey).update('storefront-auth signing key check').digest('hex'); diff --git a/examples/storefront-auth/modules/auth/src/server.ts b/examples/storefront-auth/modules/auth/src/server.ts index 0d934224c..328844b03 100644 --- a/examples/storefront-auth/modules/auth/src/server.ts +++ b/examples/storefront-auth/modules/auth/src/server.ts @@ -5,21 +5,13 @@ import { serve } from '@prisma/composer/service-rpc'; import { SQL } from 'bun'; +import { keyCheckValue } from './key-check.ts'; import service from './service.ts'; const { db } = service.load(); // db: RawPostgresConfig — the app owns its client const { signingKey } = service.input(); // signingKey: SecretString — redacts everywhere but expose() (ADR-0042) const port = service.port(); -// The E2E's KNOWN test marker for the secret the root binds to AUTH_SIGNING_SECRET -// (matched by the value the deploy provisions — see .github/workflows/e2e-deploy.yml -// and scripts/e2e-verify.sh). `secretCheck` below proves the secret round-tripped -// (root envSecret -> input-document $secret pointer -> boot lookup -> SecretBox) by comparing -// `signingKey.expose()` to this marker and returning ONLY a boolean. `.expose()` is -// the sole reader; the value is never rendered or logged (SecretBox redacts). This -// constant is a non-sensitive demonstration marker, not a real credential. -const EXPECTED_SIGNING_SECRET = 'sk_test_ci_storefront_auth'; - // The app constructs its own client from the binding (ADR-0015). Module-scoped, // so it is one pool per process. idleTimeout closes the pooled connection // before Compute's scale-to-zero drops it, so the next request reconnects @@ -47,9 +39,7 @@ const handler = serve(service, { return { ok: false }; } }, - // True iff the injected secret matches the expected marker — proof it was - // provisioned and double-looked-up, without ever returning the value. - secretCheck: async () => ({ ok: signingKey.expose() === EXPECTED_SIGNING_SECRET }), + keyCheck: async () => ({ value: keyCheckValue(signingKey.expose()) }), }, }); export default handler; diff --git a/examples/storefront-auth/modules/auth/testing/fake.ts b/examples/storefront-auth/modules/auth/testing/fake.ts index 25216b424..b40582d6f 100644 --- a/examples/storefront-auth/modules/auth/testing/fake.ts +++ b/examples/storefront-auth/modules/auth/testing/fake.ts @@ -27,9 +27,6 @@ const fakeAuth = compute({ export default serve(fakeAuth, { rpc: { verify: async ({ token }) => ({ ok: token.length > 0 }), - // The fake stands in for the auth dependency's wiring, not its secret: it - // always proves the round trip. The real secret injection is proven by the - // live auth service in the CI E2E. - secretCheck: async () => ({ ok: true }), + keyCheck: async () => ({ value: 'fake-key-check' }), }, }); diff --git a/examples/storefront-auth/modules/storefront/app/page.integration.test.ts b/examples/storefront-auth/modules/storefront/app/page.integration.test.ts index d86782f3e..58c9a2b43 100644 --- a/examples/storefront-auth/modules/storefront/app/page.integration.test.ts +++ b/examples/storefront-auth/modules/storefront/app/page.integration.test.ts @@ -46,7 +46,7 @@ function bootStandaloneNext(build: NextjsBuildAdapter): () => Promise { } describe('storefront -> auth round trip, driven over real HTTP (bootstrapService)', () => { - it('renders auth.verify() -> { ok: true } served by the fake auth on a loopback port', async () => { + it('renders auth.verify() and auth.keyCheck() served by the fake auth on a loopback port', async () => { if (!isNextjsBuild(storefrontService.build)) { throw new Error('expected the storefront service to use the nextjs build adapter'); } @@ -60,11 +60,11 @@ describe('storefront -> auth round trip, driven over real HTTP (bootstrapService ); const res = await app.fetch(new Request(app.url)); - // React separates the static text from the {String(ok)} expression with an - // empty `` comment; assert around it rather than stripping HTML. + // React separates the static text from each expression with an empty + // `` comment; assert around it rather than stripping HTML. const html = await res.text(); expect(html).toContain('Auth /verify says: true'); - expect(html).toContain('Secret /check says: true'); + expect(html).toContain('Auth /keyCheck says: fake-key-check'); }); }); diff --git a/examples/storefront-auth/modules/storefront/app/page.test.tsx b/examples/storefront-auth/modules/storefront/app/page.test.tsx index b7146b91b..8e7484964 100644 --- a/examples/storefront-auth/modules/storefront/app/page.test.tsx +++ b/examples/storefront-auth/modules/storefront/app/page.test.tsx @@ -14,7 +14,7 @@ vi.mock('../src/service.ts', async () => { default: mockService(actual.default, { auth: { verify: async ({ token }) => ({ ok: token.length > 0 }), - secretCheck: async () => ({ ok: true }), + keyCheck: async () => ({ value: 'key-check-value' }), }, }), }; @@ -27,6 +27,6 @@ describe('Home (page.tsx)', () => { const html = renderToStaticMarkup(await Home()); expect(html).toContain('Auth /verify says: true'); - expect(html).toContain('Secret /check says: true'); + expect(html).toContain('Auth /keyCheck says: key-check-value'); }); }); diff --git a/examples/storefront-auth/modules/storefront/app/page.tsx b/examples/storefront-auth/modules/storefront/app/page.tsx index 4fd2631af..2105f8a29 100644 --- a/examples/storefront-auth/modules/storefront/app/page.tsx +++ b/examples/storefront-auth/modules/storefront/app/page.tsx @@ -7,13 +7,13 @@ export const dynamic = 'force-dynamic'; export default async function Home() { const { auth } = service.load(); const { ok } = await auth.verify({ token: 'demo-token' }); - const { ok: secretOk } = await auth.secretCheck({}); + const { value: keyCheck } = await auth.keyCheck({}); return (

Storefront

Auth /verify says: {String(ok)}

-

Secret /check says: {String(secretOk)}

+

Auth /keyCheck says: {keyCheck}

); } diff --git a/examples/storefront-auth/scripts/e2e-verify.sh b/examples/storefront-auth/scripts/e2e-verify.sh deleted file mode 100644 index 9532695f5..000000000 --- a/examples/storefront-auth/scripts/e2e-verify.sh +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env bash -# Resolves the storefront service's deployed URL via the Management API (state -# is hosted, not local files), then polls it until the page renders the live -# storefront->auth round trip. Retries because a version cold-starts after -# deploy (PRO-200) and auth's DB ping can transiently fail right after idle -# (FT-5219), recovering on the next hit. Run from examples/storefront-auth. -# Requires PRISMA_SERVICE_TOKEN; STACK_NAME optionally overrides the -# project name (defaults to storefront-auth, matching the stack name the CLI deploys). -set -euo pipefail - -stack="${STACK_NAME:-storefront-auth}" -api="https://api.prisma.io/v1" -auth_header="Authorization: Bearer ${PRISMA_SERVICE_TOKEN:?PRISMA_SERVICE_TOKEN is required}" - -project_id="$(curl -sS -H "$auth_header" "$api/projects?limit=100" \ - | node -e "let d='';process.stdin.on('data',(c)=>{d+=c}).on('end',()=>{const p=(JSON.parse(d).data??[]).find((x)=>x.name===process.argv[1]);console.log(p?.id??'')})" "$stack")" -[ -n "$project_id" ] || { echo "No project named '$stack' in the workspace."; exit 1; } - -# The post-promote endpoint domain is the servable one (the create-time domain -# is a placeholder — PRO-200); by the time this script runs, deploy + promote -# have completed, so the app read returns the real domain. `/v1/apps` is the -# current Management API surface for what used to be `/v1/compute-services` -# (same underlying resources — see cold-start-canary.ts and gotchas.md PRO-217). -domain="$(curl -sS -H "$auth_header" "$api/apps?projectId=$project_id&limit=100" \ - | node -e "let d='';process.stdin.on('data',(c)=>{d+=c}).on('end',()=>{const s=(JSON.parse(d).data??[]).find((x)=>x.logicalId===process.argv[1]);console.log(s?.appEndpointDomain??'')})" "$stack.storefront")" -[ -n "$domain" ] || { echo "Project $project_id has no '$stack.storefront' app with an endpoint domain."; exit 1; } -# appEndpointDomain arrives WITH the https:// scheme (see the PRO-200 -# gotcha's captured responses); tolerate either shape. -case "$domain" in - http://*|https://*) url="$domain" ;; - *) url="https://$domain/" ;; -esac -echo "Storefront URL: $url" - -deadline=$((SECONDS + 180)) -body="" -while [ "$SECONDS" -lt "$deadline" ]; do - body="$(curl -sS --max-time 30 "$url" || true)" - clean="$(printf '%s' "$body" | sed -e 's///g' -e 's/"/"/g')" - if printf '%s' "$clean" | grep -q 'Auth /verify says: true' \ - && printf '%s' "$clean" | grep -q 'Secret /check says: true'; then - echo 'Round trip OK — storefront rendered auth.verify() -> { ok: true } AND the secret proof (secretCheck -> { ok: true })' - exit 0 - fi - sleep 6 -done -echo "Round trip never rendered both 'Auth /verify says: true' and 'Secret /check says: true' within the deadline. Last body:" -printf '%s' "$body" | head -c 3000 -exit 1 diff --git a/examples/storefront-auth/scripts/e2e-verify.ts b/examples/storefront-auth/scripts/e2e-verify.ts new file mode 100644 index 000000000..1a7623c93 --- /dev/null +++ b/examples/storefront-auth/scripts/e2e-verify.ts @@ -0,0 +1,85 @@ +#!/usr/bin/env bun +/** + * Polls the deployed storefront until `auth.verify()` returns true and + * `auth.keyCheck()` matches the key check of AUTH_SIGNING_SECRET. + * Env: AUTH_SIGNING_SECRET; PRISMA_SERVICE_TOKEN unless STOREFRONT_URL is set; + * STACK_NAME overrides the project name (default storefront-auth). + */ + +import { keyCheckValue } from '@storefront-auth/auth/key-check'; +import { type } from 'arktype'; + +const API = 'https://api.prisma.io/v1'; +const POLL_DEADLINE_MS = 180_000; +const POLL_INTERVAL_MS = 6_000; + +const Projects = type({ data: type({ id: 'string', name: 'string' }).array() }); +const Apps = type({ + data: type({ logicalId: 'string', 'appEndpointDomain?': 'string | null' }).array(), +}); + +function fail(message: string): never { + console.error(message); + process.exit(1); +} + +function requiredEnv(name: string): string { + const value = process.env[name]; + if (value === undefined || value.length === 0) fail(`${name} is required`); + return value; +} + +async function getJson(path: string, token: string): Promise { + const response = await fetch(`${API}${path}`, { headers: { Authorization: `Bearer ${token}` } }); + if (!response.ok) fail(`GET ${path} failed: ${response.status} ${response.statusText}`); + return response.json(); +} + +async function storefrontUrl(): Promise { + const override = process.env['STOREFRONT_URL']; + if (override !== undefined && override.length > 0) return override; + + const token = requiredEnv('PRISMA_SERVICE_TOKEN'); + const stack = process.env['STACK_NAME'] ?? 'storefront-auth'; + const projects = Projects.assert(await getJson('/projects?limit=100', token)); + const project = projects.data.find((p) => p.name === stack); + if (project === undefined) fail(`No project named '${stack}' in the workspace.`); + + const apps = Apps.assert(await getJson(`/apps?projectId=${project.id}&limit=100`, token)); + const domain = apps.data.find((a) => a.logicalId === `${stack}.storefront`)?.appEndpointDomain; + if (domain === undefined || domain === null || domain.length === 0) { + fail(`Project ${project.id} has no '${stack}.storefront' app with an endpoint domain.`); + } + return /^https?:\/\//.test(domain) ? domain : `https://${domain}/`; +} + +const expectedKeyCheck = keyCheckValue(requiredEnv('AUTH_SIGNING_SECRET')); + +const url = await storefrontUrl(); +console.log(`Storefront URL: ${url}`); + +const deadline = Date.now() + POLL_DEADLINE_MS; +let lastBody = ''; +while (Date.now() < deadline) { + try { + const response = await fetch(url, { signal: AbortSignal.timeout(30_000) }); + lastBody = (await response.text()).replaceAll('', ''); + if ( + lastBody.includes('Auth /verify says: true') && + lastBody.includes(`Auth /keyCheck says: ${expectedKeyCheck}`) + ) { + console.log( + 'Round trip OK: auth.verify() returned true and auth.keyCheck() matches AUTH_SIGNING_SECRET.', + ); + process.exit(0); + } + } catch (error) { + lastBody = error instanceof Error ? error.message : String(error); + } + await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL_MS)); +} +console.error( + "The storefront never rendered 'Auth /verify says: true' and the expected key check within the deadline. Last body:", +); +console.error(lastBody.slice(0, 3000)); +process.exit(1); diff --git a/examples/storefront-auth/tsconfig.json b/examples/storefront-auth/tsconfig.json index 15732ebf4..2ce9d4a8d 100644 --- a/examples/storefront-auth/tsconfig.json +++ b/examples/storefront-auth/tsconfig.json @@ -3,5 +3,5 @@ "compilerOptions": { "types": ["bun"] }, - "include": ["module.ts", "module.test.ts"] + "include": ["module.ts", "module.test.ts", "scripts"] } diff --git a/packages/0-framework/1-core/core/src/__tests__/invariants.test.ts b/packages/0-framework/1-core/core/src/__tests__/invariants.test.ts index e99a76f12..df8aa45e8 100644 --- a/packages/0-framework/1-core/core/src/__tests__/invariants.test.ts +++ b/packages/0-framework/1-core/core/src/__tests__/invariants.test.ts @@ -140,8 +140,8 @@ describe("invariant 2: the '.' authoring entry bundles lean", () => { }); }); -describe('invariant 4: core touches the environment only to read back the container transport it wrote', () => { - test("the process-env token appears only in control/deploy.ts (deserializeContainers(config, process.env), read-back for the parent→child container transport core itself owns — never a target's own vars)", () => { +describe('invariant 4: core touches the environment only to read back the transports it wrote', () => { + test("only the deploy's second process reads the process environment", () => { const sources = shippedSources(); expect(sources.length).toBeGreaterThan(0); @@ -151,7 +151,7 @@ describe('invariant 4: core touches the environment only to read back the contai return count > 0 ? [{ file, count }] : []; }); - expect(hits).toEqual([{ file: 'control/deploy.ts', count: 2 }]); + expect(hits).toEqual([{ file: 'control/deploy.ts', count: 3 }]); }); }); diff --git a/packages/0-framework/1-core/core/src/__tests__/lowering.test.ts b/packages/0-framework/1-core/core/src/__tests__/lowering.test.ts index d5c703dda..d8d50bd5d 100644 --- a/packages/0-framework/1-core/core/src/__tests__/lowering.test.ts +++ b/packages/0-framework/1-core/core/src/__tests__/lowering.test.ts @@ -16,6 +16,7 @@ import { type Bundle, buildConfig, connectionInputs, + deploymentResult, joinDeployment, type LowerContext, LowerError, @@ -42,6 +43,7 @@ import { resource, service, } from '../node.ts'; +import { PREFLIGHT_FINDINGS_ENV_VAR } from '../preflight-transport.ts'; import { resolve } from '../value.ts'; import { anyInputSchema, conn, providerContract } from './helpers.ts'; @@ -1439,6 +1441,44 @@ describe('joinDeployment', () => { }); }); +const withEnv = (values: Record, fn: () => T): T => { + const previous = new Map(Object.keys(values).map((k) => [k, process.env[k]])); + for (const [k, v] of Object.entries(values)) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + try { + return fn(); + } finally { + for (const [k, v] of previous) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + } +}; + +describe('deploymentResult', () => { + test('joins the deployed entries with the graph and carries the findings it is given', () => { + const root = module('shop', {}, (h) => { + h.provision(dbResource(), { id: 'db' }); + return {}; + }); + const unset = { + name: 'MAIL_KEY', + readers: [{ kind: 'unread', node: 'shop.mail', nodeKind: 'module' }] as const, + commands: ["prisma project env add 'MAIL_KEY=' --project 'proj' --role production"], + }; + + const result = deploymentResult('shop', Load(root), [{ address: 'shop.db', entities: [] }], { + unset: [unset], + }); + + expect(result.app).toBe('shop'); + expect(result.nodes.map((node) => node.address)).toEqual(['shop.db']); + expect(result.findings).toEqual({ unset: [unset] }); + }); +}); + describe('lowering() — the report path', () => { test('without opts.report, lowering declares no action and stays sync-runnable', () => { const { config } = fakeExtension(); @@ -1449,6 +1489,18 @@ describe('lowering() — the report path', () => { // would die. `run()` is Effect.runSync. expect(run(lowering(root, config, opts(svcBundles)))).toBeUndefined(); }); + + test('findings that do not decode fail lowering before any node is lowered', () => { + const { config, calls } = fakeExtension(); + const root = singleServiceWithDbModule(); + + withEnv({ [PREFLIGHT_FINDINGS_ENV_VAR]: 'not json' }, () => { + expect(() => + run(lowering(root, config, opts({ ...svcBundles, report: () => undefined }))), + ).toThrow(/could not be read in the second process/); + }); + expect(calls).toEqual([]); + }); }); describe('lower()', () => { @@ -1552,22 +1604,6 @@ describe('container threading through lowering() (the parent→child transport r return { descriptor, config, seen }; } - const withEnv = (values: Record, fn: () => T): T => { - const previous = new Map(Object.keys(values).map((k) => [k, process.env[k]])); - for (const [k, v] of Object.entries(values)) { - if (v === undefined) delete process.env[k]; - else process.env[k] = v; - } - try { - return fn(); - } finally { - for (const [k, v] of previous) { - if (v === undefined) delete process.env[k]; - else process.env[k] = v; - } - } - }; - const dbModule = () => module('shop', {}, (h) => { h.provision( diff --git a/packages/0-framework/1-core/core/src/__tests__/preflight-transport.test.ts b/packages/0-framework/1-core/core/src/__tests__/preflight-transport.test.ts index 35b15156c..60c4941ff 100644 --- a/packages/0-framework/1-core/core/src/__tests__/preflight-transport.test.ts +++ b/packages/0-framework/1-core/core/src/__tests__/preflight-transport.test.ts @@ -1,6 +1,14 @@ import { describe, expect, test } from 'bun:test'; import { containerEnvVarName } from '../container-transport.ts'; -import { preflightEnv, preflightEnvVarName, readPreflightPayload } from '../preflight-transport.ts'; +import { + PREFLIGHT_FINDINGS_ENV_VAR, + preflightEnv, + preflightEnvVarName, + preflightFindingsEnv, + readPreflightFindings, + readPreflightPayload, + type UnsetPlatformVariable, +} from '../preflight-transport.ts'; describe('preflightEnvVarName()', () => { test('the documented mangling — the exact @prisma/composer-prisma-cloud expectation', () => { @@ -45,7 +53,13 @@ describe('preflightEnv()', () => { ['acme/widgets', 'b'], ]), ), - ).toThrow(/"acme\.widgets" and "acme\/widgets" both map to the preflight transport variable/); + ).toThrow(/"acme\.widgets" and "acme\/widgets" both use the environment variable/); + }); + + test('an extension whose variable would be the findings variable is refused, naming the reserved variable', () => { + expect(() => preflightEnv(new Map([['findings', 'a']]))).toThrow( + 'Extension id "findings" uses the environment variable "PRISMA_COMPOSER_PREFLIGHT_FINDINGS", which Composer reserves. Rename the extension.', + ); }); }); @@ -70,3 +84,32 @@ describe('readPreflightPayload()', () => { ).toBeUndefined(); }); }); + +describe('the deploy preflight findings', () => { + const unset: UnsetPlatformVariable = { + name: 'MAIL_KEY', + readers: [{ kind: 'unread', node: 'app.mail', nodeKind: 'module' }], + commands: ["prisma project env add 'MAIL_KEY=' --project 'proj' --role production"], + }; + + test('reach the second process as the first process wrote them', () => { + expect(readPreflightFindings(preflightFindingsEnv({ unset: [unset] }))).toEqual({ + unset: [unset], + }); + }); + + test('are always written, and empty when nothing was found, so a stale value in the shell is replaced', () => { + expect(preflightFindingsEnv({ unset: [] })).toEqual({ [PREFLIGHT_FINDINGS_ENV_VAR]: '' }); + expect(readPreflightFindings(preflightFindingsEnv({ unset: [] }))).toEqual({ unset: [] }); + expect(readPreflightFindings({})).toEqual({ unset: [] }); + }); + + test('that do not decode fail rather than print a partial report', () => { + expect(() => readPreflightFindings({ [PREFLIGHT_FINDINGS_ENV_VAR]: 'not json' })).toThrow( + /could not be read in the second process/, + ); + expect(() => + readPreflightFindings({ [PREFLIGHT_FINDINGS_ENV_VAR]: '{"unset":[{"name":"A_KEY"}]}' }), + ).toThrow(/could not be read in the second process/); + }); +}); diff --git a/packages/0-framework/1-core/core/src/__tests__/progress-report.test-d.ts b/packages/0-framework/1-core/core/src/__tests__/progress-report.test-d.ts index 8724420fc..22a6f8b35 100644 --- a/packages/0-framework/1-core/core/src/__tests__/progress-report.test-d.ts +++ b/packages/0-framework/1-core/core/src/__tests__/progress-report.test-d.ts @@ -1,19 +1,13 @@ -/** - * The progress channels core hands to extensions: assemble and preflight each - * take an optional `report` callback for a bag of counts and labels, and the - * assembled artifact itself carries no measurements. Type-only (vitest - * `--typecheck`, never executed). - */ import { expectTypeOf, test } from 'vitest'; -import type { PreflightInput } from '../exports/app-config.ts'; +import type { PlatformVariableCheck, PreflightInput } from '../exports/app-config.ts'; import type { AssembleInput, Bundle } from '../exports/deploy.ts'; type ProgressReport = ((data: Readonly>) => void) | undefined; -test('assemble and preflight report through the same optional callback', () => { +test('assemble reports counts and labels, preflight what it checked and found', () => { expectTypeOf().toEqualTypeOf(); expectTypeOf().toEqualTypeOf< - ((counts: Readonly>) => void) | undefined + ((report: PlatformVariableCheck) => void) | undefined >(); }); diff --git a/packages/0-framework/1-core/core/src/control/app-config.ts b/packages/0-framework/1-core/core/src/control/app-config.ts index 6a083e5f8..186411b77 100644 --- a/packages/0-framework/1-core/core/src/control/app-config.ts +++ b/packages/0-framework/1-core/core/src/control/app-config.ts @@ -7,7 +7,7 @@ import type { } from '../container-transport.ts'; import type { Graph } from '../graph.ts'; import type { ServiceNode } from '../node.ts'; -import type { PreflightPayload } from '../preflight-transport.ts'; +import type { PreflightFindings, PreflightPayload } from '../preflight-transport.ts'; import type { AlchemyStateLayer, ApplicationDescriptor, @@ -32,10 +32,19 @@ export { containerEnvVarName, deserializeContainers, } from '../container-transport.ts'; -export type { PreflightPayload } from '../preflight-transport.ts'; +export type { + PlatformVariableReader, + PreflightFindings, + PreflightPayload, + UnsetPlatformVariable, +} from '../preflight-transport.ts'; export { + PREFLIGHT_FINDINGS_ENV_VAR, + platformVariableReaderText, preflightEnv, preflightEnvVarName, + preflightFindingsEnv, + readPreflightFindings, readPreflightPayload, } from '../preflight-transport.ts'; /** Re-exported because `RunOutcome` hands them to a reporter — reading this surface must not also require the deploy one. */ @@ -57,19 +66,22 @@ export interface ExtensionDescriptor { readonly application?: ApplicationDescriptor; /** The extension's Alchemy providers — merged across all configured extensions (config order). */ readonly providers?: () => Layer.Layer; + /** Refuses a loaded graph this extension cannot deploy by throwing `GraphCheckError`. The CLI calls it right after Load in every command that runs Load. Any other throw is a bug. */ + readonly checkGraph?: (graph: Graph) => void; /** * Deploy-time prerequisite check — the CLI runs it once, after the app's * Project/Branch are resolved and BEFORE any stack file is written or Alchemy * runs. A target uses it to verify platform prerequisites (e.g. that every - * secret env var in the provision manifest exists for the resolved environment) and - * throws to abort the deploy. Async: it talks to the platform (ADR-0029). + * platform variable a service needs has a value) and throws to abort the + * deploy. Async: it talks to the platform (ADR-0064). * * What it returns is carried to the alchemy process for this same extension * to read back there (preflight-transport.ts) — preflight runs in the CLI * parent, and the alchemy child re-imports the config from scratch, so * anything the lowering needs from it must ride that transport. The payload * is the extension's own opaque string; the framework never reads it, and a - * secret value must never go in it. + * secret value must never go in it. What it hands `input.report` reaches the + * deploy report through the framework's own transport, not this payload. * * METHOD SYNTAX REQUIRED, for the same reason ContainerDescriptor's members * need it: the framework hands over the erased `PreflightInput`, @@ -134,18 +146,23 @@ export interface PreflightInput { readonly container: ContainerInstance | undefined; /** The environment name. */ readonly environment: string; - /** The app configuration the command read for this environment. Preflight fills a missing `envParam()` param's platform variable from it. */ + /** The app configuration the command read for this environment. */ readonly configuration: AppConfiguration; - /** The app secrets the command read for this environment. Preflight fills a missing `envSecret()` platform variable from it. */ + /** The app secrets the command read for this environment. */ readonly secrets: AppSecrets; /** What the caller has already authenticated, for the platform calls preflight makes. Absent means the extension falls back to its own credential protocol. */ readonly credentials?: ContainerCredentials | undefined; - /** Hands the deploy's progress output what the check covered, as counts (e.g. `{ checked: 53, filled: 0, missing: 0 }`). Counts only: never a name or a value. */ - readonly report?: ((counts: Readonly>) => void) | undefined; + /** Hands the deploy how many platform variables the check read and which have no value. */ + readonly report?: ((check: PlatformVariableCheck) => void) | undefined; /** Shows the user a warning that does not stop the run. Only `prisma dev` passes it and shows the warnings; deploy passes none. */ readonly warn?: ((message: string) => void) | undefined; } +/** What a deploy preflight's check of platform variables found: how many it read, and the ones with no value that nothing requires. */ +export interface PlatformVariableCheck extends PreflightFindings { + readonly checked: number; +} + /** The resolved destroy context handed to an extension's `teardown` hook. */ export interface TeardownInput { /** The calling extension's own resolved container; `undefined` when it declares no container descriptor. Narrow with the extension's guard. */ @@ -332,7 +349,7 @@ export interface LocalTargetProvidersInput { readonly devDir: string; /** The text of each configuration key the graph reads, from the app configuration file. A local deployment takes the text its param rows point at from it. */ readonly configurationKeyText: Readonly>; - /** The app secrets the graph's `envSecret()`s name, which the first process handed the second. A local deployment takes the text its `$secret` pointers name from it. */ + /** The app secrets the graph's `envSecret()`s name, which the first process handed the second. A local deployment takes the text its `$platform` markers name from it. */ readonly namedSecrets: NamedSecrets; } @@ -457,10 +474,10 @@ export type ConfigAdapter = (input: ConfigAdapterInput) => ConfigValues | Promis /** The app configuration (ADR-0062): what the `config` adapters return, merged by top-level key, a later adapter winning. It is the root's input, and what `envParam()` reads. */ export type AppConfiguration = Readonly>; -/** The app secrets (ADR-0062): what the `secrets` adapters return, merged the same way. Every value is text. Preflight fills a missing `envSecret()` platform variable from it, and `prisma dev` hands its second process the values the graph's `envSecret()`s name. Composer redacts the values the graph's `envSecret()`s name wherever it prints. */ +/** The app secrets (ADR-0062): what the `secrets` adapters return, merged the same way. Every value is text. Nothing fills a platform variable from them. `prisma dev` hands its second process the values under the secrets keys of the graph's secret sources, and Composer redacts those values wherever it prints. */ export type AppSecrets = Readonly>; -/** The app secrets an `envSecret()` in the graph names, each with text. `prisma dev`'s first process hands them to its second process in `PRISMA_COMPOSER_NAMED_SECRETS`. */ +/** The app secrets under the secrets keys of the graph's secret sources, each with text. `prisma dev`'s first process hands them to its second process in `PRISMA_COMPOSER_NAMED_SECRETS`. */ export type NamedSecrets = Readonly>; /** diff --git a/packages/0-framework/1-core/core/src/control/deploy.ts b/packages/0-framework/1-core/core/src/control/deploy.ts index 57aba6f0e..0980cd4ea 100644 --- a/packages/0-framework/1-core/core/src/control/deploy.ts +++ b/packages/0-framework/1-core/core/src/control/deploy.ts @@ -20,6 +20,7 @@ import { type ServiceNode, slotsOf, } from '../node.ts'; +import { type PreflightFindings, readPreflightFindings } from '../preflight-transport.ts'; import type { ContainerInstance, ExtensionDescriptor, @@ -210,6 +211,8 @@ export interface DeployedNode { export interface DeploymentResult { readonly app: string; readonly nodes: readonly DeployedNode[]; + /** What every extension's deploy preflight found that did not stop the deploy. */ + readonly findings: PreflightFindings; } export interface LowerOptions { @@ -495,6 +498,16 @@ export function joinDeployment( return nodes; } +/** The deploy report: every deployed node joined with its graph node, and the findings the CLI carried from the deploy preflight. */ +export function deploymentResult( + app: string, + graph: Graph, + entries: readonly { address: string; entities: readonly DeployedEntity[] }[], + findings: PreflightFindings, +): DeploymentResult { + return { app, nodes: joinDeployment(graph, entries), findings }; +} + function missingBundleError(id: NodeId): LowerError { return new LowerError( `No bundle provided for service "${id}" (opts.bundles["${id}"] is required).`, @@ -686,6 +699,7 @@ export function lowering( // framework transport (the CLI parent wrote it; this is the alchemy // child) — read-only for the rest of this lowering. const containers = deserializeContainers(config.extensions, process.env); + const findings = readPreflightFindings(process.env); const lowered = new Map(); // Each node's reported entities, in topo order — the loop is the only // party that holds both the node's identity and what it became. Collected @@ -878,7 +892,7 @@ export function lowering( // strings here. The graph rides in on the closure, never in the // input: the plan hashes the resolved input, and a node carries // functions and Standard Schemas. - report({ app: opts.name, nodes: joinDeployment(graph, input.entries) }); + report(deploymentResult(opts.name, graph, input.entries, findings)); }), ); // `Date.now()` forces the report to run on an otherwise unchanged diff --git a/packages/0-framework/1-core/core/src/exports/index.ts b/packages/0-framework/1-core/core/src/exports/index.ts index 43db4cb36..cfaba2213 100644 --- a/packages/0-framework/1-core/core/src/exports/index.ts +++ b/packages/0-framework/1-core/core/src/exports/index.ts @@ -44,6 +44,7 @@ export type { } from '../graph.ts'; export { assertAppName, + GraphCheckError, Load, LoadError, RESOURCE_OUT_PORT, diff --git a/packages/0-framework/1-core/core/src/graph-types.ts b/packages/0-framework/1-core/core/src/graph-types.ts index c6b4a8ec7..a3a8a5d3c 100644 --- a/packages/0-framework/1-core/core/src/graph-types.ts +++ b/packages/0-framework/1-core/core/src/graph-types.ts @@ -186,3 +186,14 @@ export class LoadError extends CliStructuredError { super('COMPOSE.GRAPH_INVALID', message, options); } } + +/** What an extension's `checkGraph` throws to refuse a graph it cannot deploy. Any other throw from the hook is a bug (ADR-0044). The CLI adds the refusing extension's id as `meta.extension`. */ +export class GraphCheckError extends CliStructuredError { + constructor(message: string, options?: { readonly extension?: string }) { + super( + 'COMPOSE.GRAPH_INVALID', + message, + options?.extension === undefined ? undefined : { meta: { extension: options.extension } }, + ); + } +} diff --git a/packages/0-framework/1-core/core/src/graph.ts b/packages/0-framework/1-core/core/src/graph.ts index 853087def..3a3103c49 100644 --- a/packages/0-framework/1-core/core/src/graph.ts +++ b/packages/0-framework/1-core/core/src/graph.ts @@ -24,7 +24,7 @@ export type { RootInputField, SecretSourceUse, } from './graph-types.ts'; -export { LoadError, RESOURCE_OUT_PORT } from './graph-types.ts'; +export { GraphCheckError, LoadError, RESOURCE_OUT_PORT } from './graph-types.ts'; const APP_NAME = /^[A-Za-z0-9][A-Za-z0-9_-]*$/; diff --git a/packages/0-framework/1-core/core/src/preflight-transport.ts b/packages/0-framework/1-core/core/src/preflight-transport.ts index 03cc873df..22ed575bb 100644 --- a/packages/0-framework/1-core/core/src/preflight-transport.ts +++ b/packages/0-framework/1-core/core/src/preflight-transport.ts @@ -25,11 +25,15 @@ export function preflightEnv(payloads: ReadonlyMap): Record> => + typeof value === 'object' && value !== null && !Array.isArray(value); + +const isStringArray = (value: unknown): value is readonly string[] => + Array.isArray(value) && value.every((member) => typeof member === 'string'); + +function isPlatformVariableReader(value: unknown): value is PlatformVariableReader { + if (!isRecord(value)) return false; + switch (value['kind']) { + case 'field': + return typeof value['service'] === 'string' && typeof value['path'] === 'string'; + case 'param': + return typeof value['service'] === 'string' && typeof value['param'] === 'string'; + case 'unread': + return ( + typeof value['node'] === 'string' && + (value['nodeKind'] === 'module' || value['nodeKind'] === 'resource') + ); + default: + return false; + } +} + +function isUnsetPlatformVariable(value: unknown): value is UnsetPlatformVariable { + return ( + isRecord(value) && + typeof value['name'] === 'string' && + Array.isArray(value['readers']) && + value['readers'].every(isPlatformVariableReader) && + isStringArray(value['commands']) + ); +} + +function isPreflightFindings(value: unknown): value is PreflightFindings { + return ( + isRecord(value) && + Array.isArray(value['unset']) && + value['unset'].every(isUnsetPlatformVariable) + ); +} + +/** The variable that carries every extension's deploy preflight findings from the first process to the second. No extension's preflight variable may take this name. */ +export const PREFLIGHT_FINDINGS_ENV_VAR = 'PRISMA_COMPOSER_PREFLIGHT_FINDINGS'; + +/** The env entry the CLI sets on the second process. It is always set, and empty when nothing was found, so a stale value in the shell never reads as this run's. */ +export function preflightFindingsEnv(findings: PreflightFindings): Record { + return { + [PREFLIGHT_FINDINGS_ENV_VAR]: findings.unset.length === 0 ? '' : JSON.stringify(findings), + }; +} + +/** The second process's side: the findings the CLI carried, or none. Throws on a value that does not decode: that is a framework bug. */ +export function readPreflightFindings( + env: Readonly>, +): PreflightFindings { + const text = env[PREFLIGHT_FINDINGS_ENV_VAR]; + if (text === undefined || text.length === 0) return { unset: [] }; + let parsed: unknown; + try { + parsed = JSON.parse(text); + } catch { + parsed = undefined; + } + if (!isPreflightFindings(parsed)) { + throw new Error( + `The deploy check's findings could not be read in the second process (${PREFLIGHT_FINDINGS_ENV_VAR}). This is a Composer bug; deploying again will not fix it.`, + ); + } + return parsed; +} diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/pipeline.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/pipeline.test.ts index a46037362..a05143358 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/pipeline.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/pipeline.test.ts @@ -1,6 +1,7 @@ import { describe, expect, test } from 'bun:test'; import * as path from 'node:path'; -import { runPipeline } from '../pipeline.ts'; +import { type Graph, GraphCheckError } from '@internal/core'; +import { resolveAppIdentity, runPipeline } from '../pipeline.ts'; import { SecretRedaction } from '../redaction.ts'; import prismaConfig from './fixtures/pipeline/prisma.config.ts'; import appConfigurationConfig from './fixtures/pipeline-app-configuration/prisma.config.ts'; @@ -159,3 +160,110 @@ describe('runPipeline()', () => { expect([...redaction.knownSecrets.values()]).toContain('from-secrets'); }); }); + +describe('an extension graph check', () => { + const composer = prismaConfig.composer; + if (composer === undefined) throw new Error('the fixture declares a composer section'); + const file = path.join(fixtureDir, 'prisma.config.ts'); + const [fixtureExtension, ...otherExtensions] = composer.extensions; + if (fixtureExtension === undefined) throw new Error('the fixture declares an extension'); + + const withCheck = (checkGraph: (graph: Graph) => void, others = otherExtensions) => ({ + value: { ...composer, extensions: [{ ...fixtureExtension, checkGraph }, ...others] }, + file, + }); + + test('runPipeline hands every graph check the loaded graph', async () => { + const checked: Graph[] = []; + + const { graph } = await runPipeline( + undefined, + undefined, + 'staging', + fixtureDir, + withCheck((loaded) => checked.push(loaded)), + noAssembly, + ); + + expect(checked).toEqual([graph]); + }); + + test('a graph check that refuses the graph stops the command before anything is built, with its message', async () => { + let assembled = false; + const uncovered = withCheck(() => { + throw new GraphCheckError('the graph is wrong'); + }, []); + + const run = runPipeline(undefined, undefined, 'staging', fixtureDir, uncovered, { + runAssembler: async () => { + assembled = true; + return { dir: '', entry: '' }; + }, + }); + + await expect(run).rejects.toMatchObject({ + code: 'COMPOSE.GRAPH_INVALID', + message: 'the graph is wrong', + meta: { extension: fixtureExtension.id }, + }); + expect(assembled).toBe(false); + }); + + test('any other throw from a graph check is a bug, and reaches the caller unchanged', async () => { + const bug = new TypeError('cannot read properties of undefined'); + const broken = withCheck(() => { + throw bug; + }); + + const run = runPipeline(undefined, undefined, 'staging', fixtureDir, broken, noAssembly); + + expect(await run.catch((error: unknown) => error)).toBe(bug); + }); + + test('a graph check runs with the app secrets already known to redaction', async () => { + const appComposer = appConfigurationConfig.composer; + if (appComposer === undefined) throw new Error('the fixture declares a composer section'); + const [appExtension, ...appOthers] = appComposer.extensions; + if (appExtension === undefined) throw new Error('the fixture declares an extension'); + const redaction = new SecretRedaction({}); + let knownAtCheck: string[] = []; + + await runPipeline( + undefined, + undefined, + 'staging', + appConfigurationDir, + { + value: { + ...appComposer, + extensions: [ + { + ...appExtension, + checkGraph: () => { + knownAtCheck = [...redaction.knownSecrets.values()]; + }, + }, + ...appOthers, + ], + config: [() => ({ CONFIG_POOL_PROBE: 'probe' })], + secrets: [() => ({ S11_SECRET: 'from-secrets' })], + }, + file: path.join(appConfigurationDir, 'prisma.config.ts'), + }, + { ...noAssembly, readSecrets: true, redaction }, + ); + + expect(knownAtCheck).toContain('from-secrets'); + }); + + test('resolveAppIdentity runs no graph check', async () => { + let called = false; + const source = withCheck(() => { + called = true; + }); + + await resolveAppIdentity(undefined, undefined, fixtureDir, source); + + expect(called).toBe(false); + }); +}); diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/redaction.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/redaction.test.ts index 35291e2f4..903330295 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/redaction.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/redaction.test.ts @@ -51,6 +51,21 @@ describe('SecretRedaction', () => { expect(redaction.knownSecrets.values().size).toBe(0); }); + test('SecretRedaction keeps a placeholder after a name that contains SECRET', () => { + const text = "set 'AUTH_SIGNING_SECRET=' first"; + + expect(new SecretRedaction({}).redact(text)).toBe(text); + expect(new SecretRedaction({}).redact('AUTH_SIGNING_SECRET=sk_live_abc')).toBe( + 'AUTH_SIGNING_SECRET=[redacted]', + ); + }); + + test('SecretRedaction redacts a real value that only starts with the placeholder', () => { + for (const text of ['SECRET=abc123', 'SECRET=-real', 'password=s3cret']) { + expect(new SecretRedaction({}).redact(text)).not.toContain(''); + } + }); + test('SecretRedaction hides a value added after it first redacted', () => { const redaction = knowing('first-secret'); expect(redaction.redact('first-secret later-secret')).toBe('[redacted] later-secret'); diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/render-deployment.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/render-deployment.test.ts index 9aa440aa3..ec2b8b5d1 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/render-deployment.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/render-deployment.test.ts @@ -30,7 +30,11 @@ const deployed = (address: string, entities: DeployedNode['entities']): Deployed entities, }); -const result = (app: string, nodes: readonly DeployedNode[]): DeploymentResult => ({ app, nodes }); +const result = (app: string, nodes: readonly DeployedNode[]): DeploymentResult => ({ + app, + nodes, + findings: { unset: [] }, +}); describe('renderDeployment', () => { test('renders the pinned tree: nested addresses, aligned entities, urls on their own line', () => { @@ -103,7 +107,7 @@ describe('renderDeployment', () => { id: 'cps_1', url: 'https://a.example', details: { - input: '{"greeting":"hello","stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + input: '{"greeting":"hello","stripeKey":$platform STRIPE_SECRET_KEY}', absent: 'greeting → ENV_GREETING\nregion → ENV_REGION', }, }, @@ -115,7 +119,7 @@ describe('renderDeployment', () => { 'app', '└─ web compute-service cps_1', ' https://a.example', - ' input {"greeting":"hello","stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + ' input {"greeting":"hello","stripeKey":$platform STRIPE_SECRET_KEY}', ' absent greeting → ENV_GREETING', ' absent region → ENV_REGION', ].join('\n'), @@ -151,6 +155,48 @@ describe('renderDeployment', () => { expect(renderDeployment(result('app', []))).toBe('app'); }); + test('ends with every platform variable that has no value, its readers and the commands that set it', () => { + const rendered = renderDeployment({ + ...result('shop', [deployed('shop.email', [{ kind: 'compute-service', id: 'cps_1' }])]), + findings: { + unset: [ + { + name: 'EMAIL_API_KEY', + readers: [ + { kind: 'field', service: 'shop.email', path: 'apiKey' }, + { kind: 'param', service: 'shop.email', param: 'key' }, + ], + commands: [ + "prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview", + "prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview --branch 'staging'", + ], + }, + { + name: 'AUDIT_TOKEN', + readers: [{ kind: 'unread', node: 'shop.audit', nodeKind: 'module' }], + commands: [ + "prisma project env add 'AUDIT_TOKEN=' --project 'proj_1' --role production", + ], + }, + ], + }, + }); + + expect(rendered).toBe( + [ + 'shop', + '└─ email compute-service cps_1', + '', + 'Platform variables with no value:', + ' - EMAIL_API_KEY (read by service "shop.email", field "apiKey"; read by service "shop.email", per-key param "key")', + " prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview", + " prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview --branch 'staging'", + ' - AUDIT_TOKEN (named by module "shop.audit", read by no service)', + " prisma project env add 'AUDIT_TOKEN=' --project 'proj_1' --role production", + ].join('\n'), + ); + }); + test('deep nesting keeps every entity in one column', () => { const results = [ deployed('app.a.b.c', [{ kind: 'compute-service', id: 'cps_1' }]), diff --git a/packages/0-framework/3-tooling/cli/src/exports/control.ts b/packages/0-framework/3-tooling/cli/src/exports/control.ts index 46f3c39e9..b11f00bb0 100644 --- a/packages/0-framework/3-tooling/cli/src/exports/control.ts +++ b/packages/0-framework/3-tooling/cli/src/exports/control.ts @@ -14,6 +14,11 @@ * structural recognition; ADR 239 in prisma/prisma). */ +export type { + PlatformVariableReader, + PreflightFindings, + UnsetPlatformVariable, +} from '@internal/core/config'; export type { CliStructuredError } from '@internal/foundation/errors'; export type { NotOk, Ok, Result } from '@internal/foundation/result'; export type { ComposerConfigSource } from '../composer-config.ts'; diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/deploy.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/deploy.test.ts index c1e108b28..75a6c3cec 100644 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/deploy.test.ts +++ b/packages/0-framework/3-tooling/cli/src/family/__tests__/deploy.test.ts @@ -445,7 +445,22 @@ describe('step events', () => { step: { name: 'preflight' }, outcome: 'ok', durationMs: 38_170, - data: { checked: 53, filled: 0, missing: 0 }, + data: { checked: 53, unset: 1 }, + }, + { + kind: 'preflight-findings', + findings: { + unset: [ + { + name: 'EMAIL_API_KEY', + readers: [{ kind: 'field', service: 'shop.email', path: 'apiKey' }], + commands: [ + "prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview", + "prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview --branch 'pr-7'", + ], + }, + ], + }, }, { kind: 'step-started', step: { name: 'record' } }, { kind: 'step-finished', step: { name: 'record' }, outcome: 'failed', durationMs: 850 }, @@ -482,6 +497,10 @@ describe('step events', () => { '✔ assemble services (3m 42s)', '▸ check environment variables', '✔ check environment variables (38.2s)', + 'Platform variables with no value:', + ' - EMAIL_API_KEY (read by service "shop.email", field "apiKey")', + " prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview", + " prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role preview --branch 'pr-7'", '▸ record result', '✘ record result (850ms)', 'console: https://console.ewr.prisma.build', @@ -490,6 +509,19 @@ describe('step events', () => { expect(plain(result.stderr)).toContain('Deployed shop to pr-7 in 12.3s.'); }); + test('nothing prints for findings that list no variable', async () => { + const { cli } = composerCli({ + fixtures: { + deploy, + deployEvents: [{ kind: 'preflight-findings', findings: { unset: [] } }], + }, + }); + const result = await cli.run(['deploy', '--entry', 'src/service.ts'], AS_TTY); + + expect(result.exitCode).toBe(0); + expect(plain(result.stderr)).not.toContain('Platform variables'); + }); + test("a finished step's durationMs is the measured one, even when the adapter reported its own", async () => { const { cli } = composerCli({ fixtures: { @@ -523,6 +555,7 @@ describe('step events', () => { 'step-finished', 'step-started', 'step-finished', + 'message', 'step-started', 'step-finished', 'endpoint', @@ -542,14 +575,18 @@ describe('step events', () => { }); expect(result.json[5]).toMatchObject({ id: 'deploy.preflight', - data: { durationMs: 38_170, checked: 53, filled: 0, missing: 0 }, + data: { durationMs: 38_170, checked: 53, unset: 1 }, + }); + expect(result.json[6]).toMatchObject({ + kind: 'message', + data: { unset: [{ name: 'EMAIL_API_KEY', readers: [{ service: 'shop.email' }] }] }, }); - expect(result.json[8]).toMatchObject({ + expect(result.json[9]).toMatchObject({ kind: 'endpoint', name: 'console', url: 'https://console.ewr.prisma.build', }); - expect(result.json[9]).toMatchObject({ + expect(result.json[10]).toMatchObject({ kind: 'result', envelope: { ok: true, result: { summary: { app: 'shop' }, durationMs: 12_300 } }, }); diff --git a/packages/0-framework/3-tooling/cli/src/family/commands/deploy.ts b/packages/0-framework/3-tooling/cli/src/family/commands/deploy.ts index 94568e149..13894c2c4 100644 --- a/packages/0-framework/3-tooling/cli/src/family/commands/deploy.ts +++ b/packages/0-framework/3-tooling/cli/src/family/commands/deploy.ts @@ -12,6 +12,7 @@ import { type DeployEvent, type DeployStep, } from '../../operations/deploy.ts'; +import { unsetPlatformVariablesText } from '../../unset-platform-variables.ts'; import { convergeSpawn, operationDeps, settleConverge } from '../converge.ts'; import { dataLine } from '../data-lines.ts'; import type { ComposerOperations } from '../family.ts'; @@ -45,8 +46,14 @@ function formatDuration(ms: number): string { return `${String(Math.floor(seconds / 60))}m ${String(seconds % 60)}s`; } -/** The operation's step events as engine events. The engine prints only `step`, so a finished step carries its duration in the text too. */ -function toEngineEvent(event: DeployEvent): EngineEvent { +/** The operation's events as engine events. */ +function toEngineEvent(event: DeployEvent): EngineEvent | undefined { + if (event.kind === 'preflight-findings') { + const text = unsetPlatformVariablesText(event.findings.unset); + return text === undefined + ? undefined + : { kind: 'message', severity: 'warn', text, data: event.findings }; + } const id = stepId(event.step); if (event.kind === 'step-started') { return { @@ -128,7 +135,10 @@ export const createDeployCommand = (operations: ComposerOperations) => cwd: ctx.cwd, reportPath: args.flags.report, reportId: args.flags.buildId, - onEvent: (event) => ctx.report(toEngineEvent(event)), + onEvent: (event) => { + const engineEvent = toEngineEvent(event); + if (engineEvent !== undefined) ctx.report(engineEvent); + }, }, operationDeps({ alchemy, diff --git a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts index b72cde88b..a84ae9b1c 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts @@ -19,7 +19,9 @@ import type { LocateContainerInput, PrismaAppConfig, RunOutcome, + UnsetPlatformVariable, } from '@internal/core/config'; +import { readPreflightFindings } from '@internal/core/config'; import { nodeAddresses } from '@internal/core/deploy'; import type { LocalTargetAttachment, @@ -41,8 +43,10 @@ import { engineFailureFilePath, } from '../../deployment-summary.ts'; import type { AppIdentity } from '../../pipeline.ts'; +import { renderDeployment } from '../../render-deployment.ts'; import { type AlchemyInvocation, alchemyCommandLine, spawnCommandLine } from '../../run-alchemy.ts'; import { RUN_REPORT_VERSION } from '../../run-report.ts'; +import { unsetPlatformVariablesText } from '../../unset-platform-variables.ts'; import { type DeployEvent, deployWithDeps } from '../deploy.ts'; import { type DestroyInput, destroyWithDeps } from '../destroy.ts'; import { type DevEvent, devWithDeps } from '../dev.ts'; @@ -1092,6 +1096,9 @@ describe('deploy()', () => { /** Each event as `kind name[:address] outcome`, for asserting order without timings. */ function stepLine(event: DeployEvent): string { + if (event.kind === 'preflight-findings') { + return `findings ${event.findings.unset.map((variable) => variable.name).join(', ')}`; + } const name = event.step.name === 'assemble-service' ? `assemble-service:${event.step.address}` @@ -1099,6 +1106,12 @@ function stepLine(event: DeployEvent): string { return event.kind === 'step-started' ? `started ${name}` : `finished ${name} ${event.outcome}`; } +const EMAIL_API_KEY_UNSET: UnsetPlatformVariable = { + name: 'EMAIL_API_KEY', + readers: [{ kind: 'field', service: 'shop.email', path: 'apiKey' }], + commands: ["prisma project env add 'EMAIL_API_KEY=' --project 'proj_1' --role production"], +}; + describe('deploy() step events', () => { test('a successful deploy reports every step in order, each with a duration and its data', async () => { const app = makeAppDir('hello-steps'); @@ -1110,7 +1123,7 @@ describe('deploy() step events', () => { config: composerConfig( fakeConfig({ preflight: async (input) => { - input.report?.({ checked: 3, filled: 1, missing: 0 }); + input.report?.({ checked: 3, unset: [EMAIL_API_KEY_UNSET] }); return undefined; }, }), @@ -1143,6 +1156,7 @@ describe('deploy() step events', () => { 'finished connect ok', 'started preflight', 'finished preflight ok', + 'findings EMAIL_API_KEY', 'started apply', 'finished apply ok', 'started record', @@ -1156,11 +1170,48 @@ describe('deploy() step events', () => { expect(finished('assemble-service')).toMatchObject({ data: { strategy: 'traced', filesTraced: 975, bytes: 4096 }, }); - expect(finished('preflight')).toMatchObject({ data: { checked: 3, filled: 1, missing: 0 } }); + expect(finished('preflight')).toMatchObject({ data: { checked: 3, unset: 1 } }); + expect(events).toContainEqual({ + kind: 'preflight-findings', + findings: { unset: [EMAIL_API_KEY_UNSET] }, + }); if (!result.ok) throw new Error('unreachable'); expect(result.value.durationMs).toBeGreaterThanOrEqual(0); }); + test("the deploy preflight's findings reach the second process on the framework's own variable", async () => { + const app = makeAppDir('hello-findings'); + let childEnv: Readonly> = {}; + + const result = await silently(() => + deployWithDeps( + { + config: composerConfig( + fakeConfig({ + preflight: async (input) => { + input.report?.({ checked: 1, unset: [EMAIL_API_KEY_UNSET] }); + return undefined; + }, + }), + ), + entry: app.entryPath, + environment: 'ci-7', + cwd: app.dir, + }, + { + runAssembler: fakeAssembler, + alchemy: async (invocation) => { + childEnv = invocation.env; + return { exitCode: 0, signal: null }; + }, + }, + ), + ); + + expect(result.ok).toBe(true); + expect(readPreflightFindings(childEnv)).toEqual({ unset: [EMAIL_API_KEY_UNSET] }); + }); + test('a thrown defect closes the open steps as failed and still records the result', async () => { const app = makeAppDir('hello-steps-defect'); const events: DeployEvent[] = []; @@ -3683,7 +3734,7 @@ describe('the operations read the app configuration and the app secrets from the }); test('an app secrets value appears in no event, result or report', async () => { - const app = makeAppDir(); + const app = makeAppDir('fixture-app', 'S4_SENTINEL'); const sentinel = 'never-printed-7f3a'; const events: DeployEvent[] = []; const received: string[] = []; @@ -3697,7 +3748,15 @@ describe('the operations read the app configuration and the app secrets from the preflight: async (input) => { const value = input.secrets['S4_SENTINEL']; if (value !== undefined) received.push(value); - input.report?.({ checked: 1, filled: 1, missing: 0 }); + input.report?.({ + checked: 2, + unset: [ + { + ...EMAIL_API_KEY_UNSET, + readers: [{ kind: 'field', service: 'app', path: `config.${sentinel}` }], + }, + ], + }); return undefined; }, }), @@ -3717,6 +3776,15 @@ describe('the operations read the app configuration and the app secrets from the expect(received).toEqual([sentinel]); expect(events.length).toBeGreaterThan(0); for (const event of events) expect(JSON.stringify(event)).not.toContain(sentinel); + const findings = events.find((event) => event.kind === 'preflight-findings'); + if (findings?.kind !== 'preflight-findings') throw new Error('expected the findings event'); + const block = unsetPlatformVariablesText(findings.findings.unset); + expect(block).toContain('EMAIL_API_KEY'); + expect(block).toContain('config.[redacted]'); + expect(block).not.toContain(sentinel); + expect(renderDeployment({ app: 'app', nodes: [], findings: findings.findings })).not.toContain( + sentinel, + ); expect(JSON.stringify(result)).not.toContain(sentinel); expect(fs.readFileSync(reportPath, 'utf8')).not.toContain(sentinel); }); diff --git a/packages/0-framework/3-tooling/cli/src/operations/deploy.ts b/packages/0-framework/3-tooling/cli/src/operations/deploy.ts index 0c2669315..17acd9718 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/deploy.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/deploy.ts @@ -6,6 +6,7 @@ * fails to load comes back as a structured failure, never a throw out of * the host. */ +import type { PreflightFindings } from '@internal/core/config'; import type { CliStructuredError } from '@internal/foundation/errors'; import { notOk, ok, type Result } from '@internal/foundation/result'; import type { ComposerConfigSource } from '../composer-config.ts'; @@ -26,7 +27,9 @@ export type DeployEvent = readonly durationMs: number; /** What the step measured, as its build adapter or deploy target reported it. The fields vary by adapter and target. */ readonly data?: Readonly> | undefined; - }; + } + /** Right after the preflight step, when it found platform variables with no value that no service needs. */ + | { readonly kind: 'preflight-findings'; readonly findings: PreflightFindings }; /** The environment `deploy` targets when none is named. */ export const DEFAULT_DEPLOY_ENVIRONMENT = 'production'; diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts index f9c0a0767..ce5441d4e 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts @@ -12,10 +12,17 @@ import type { Graph } from '@internal/core'; import type { ContainerCredentials, ContainerInstance, + PlatformVariableCheck, ReporterDescriptor, RunReporter, + UnsetPlatformVariable, +} from '@internal/core/config'; +import { + containerEnv, + preflightEnv, + preflightEnvVarName, + preflightFindingsEnv, } from '@internal/core/config'; -import { containerEnv, preflightEnv, preflightEnvVarName } from '@internal/core/config'; import { nodeAddresses, secretsKeys } from '@internal/core/deploy'; import { CliStructuredError } from '@internal/foundation/errors'; import { notOk, ok, okVoid, type Result } from '@internal/foundation/result'; @@ -35,7 +42,13 @@ import { } from '../deployment-summary.ts'; import { GENERATED_STACK_RELATIVE_PATH, writeStackFile } from '../generate-stack.ts'; import { type PipelineDeps, type PipelineResult, runPipeline } from '../pipeline.ts'; -import { redactError, redactSummary, redactThrown, SecretRedaction } from '../redaction.ts'; +import { + redactError, + redactSummary, + redactThrown, + redactUnsetPlatformVariable, + SecretRedaction, +} from '../redaction.ts'; import { type AlchemyOutcome, alchemyInvocation, @@ -113,6 +126,7 @@ function deploySteps(onEvent: ((event: DeployEvent) => void) | undefined) { failOpen(): void { for (const { step } of [...open.values()].reverse()) finish(step, 'failed'); }, + emit, }; } @@ -538,18 +552,17 @@ async function runStackPipelineInner( alchemyStage = containers.get(config.state.extension)?.alchemyStage ?? environment; steps.finish({ name: 'connect' }); - // Preflight (deploy only): each extension verifies its platform - // prerequisites — e.g. that every secret env var in the provision manifest - // exists for the resolved environment (ADR-0029) — BEFORE any stack file is written - // or Alchemy runs, so a missing secret fails fast with nothing side-effected. + const unset: UnsetPlatformVariable[] = []; if (action === 'deploy') { steps.start({ name: 'preflight' }); - // Summed across extensions: each reports counts of its own checks. - const counts: Record = {}; - const report = (reported: Readonly>): void => { - for (const [key, value] of Object.entries(reported)) { - counts[key] = (counts[key] ?? 0) + value; - } + let checked = 0; + const report = (reported: PlatformVariableCheck): void => { + checked += reported.checked; + unset.push( + ...reported.unset.map((variable) => + redactUnsetPlatformVariable(variable, opts.redaction), + ), + ); }; for (const extension of config.extensions) { if (extension.preflight === undefined) continue; @@ -576,7 +589,8 @@ async function runStackPipelineInner( } catch (error) { throw toStructured('DEPLOY.PREFLIGHT_FAILED', error); } - steps.finish({ name: 'preflight' }, counts); + steps.finish({ name: 'preflight' }, { checked, unset: unset.length }); + if (unset.length > 0) steps.emit({ kind: 'preflight-findings', findings: { unset } }); } // The child inherits this process's environment, so every transport var // this run did NOT produce is blanked — a stale value exported into the @@ -585,6 +599,7 @@ async function runStackPipelineInner( for (const extension of config.extensions) { preflightTransportEnv[preflightEnvVarName(extension.id)] ??= ''; } + Object.assign(preflightTransportEnv, preflightFindingsEnv({ unset })); } catch (error) { if (CliStructuredError.is(error)) return notOk(error); throw error; diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts index c2b79280b..f3ae1e3ca 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts @@ -12,7 +12,12 @@ import * as path from 'node:path'; import { isDeepStrictEqual } from 'node:util'; import type { Graph, ServiceNode } from '@internal/core'; import type { ContainerInstance, PrismaAppConfig } from '@internal/core/config'; -import { containerEnv, preflightEnv, preflightEnvVarName } from '@internal/core/config'; +import { + containerEnv, + preflightEnv, + preflightEnvVarName, + preflightFindingsEnv, +} from '@internal/core/config'; import type { DevServer } from '@internal/core/deploy'; import { nodeAddresses, secretsKeys } from '@internal/core/deploy'; import type { LocalTargetAttachment, LocalTargetDescriptor } from '@internal/core/local-target'; @@ -285,6 +290,7 @@ async function executeDevInner( for (const extension of config.extensions) { preflightTransportEnv[preflightEnvVarName(extension.id)] ??= ''; } + Object.assign(preflightTransportEnv, preflightFindingsEnv({ unset: [] })); // Emulators — ensure the daemons this topology's node kinds need, and their per-node state. for (const [id, dev] of resolved) { diff --git a/packages/0-framework/3-tooling/cli/src/pipeline.ts b/packages/0-framework/3-tooling/cli/src/pipeline.ts index 9e320918e..b6d8052cc 100644 --- a/packages/0-framework/3-tooling/cli/src/pipeline.ts +++ b/packages/0-framework/3-tooling/cli/src/pipeline.ts @@ -17,8 +17,13 @@ import { type RunAssembler, } from '@internal/assemble'; import type { Graph } from '@internal/core'; -import { assertAppName, Load } from '@internal/core'; -import type { AppConfiguration, AppSecrets, PrismaAppConfig } from '@internal/core/config'; +import { assertAppName, GraphCheckError, Load } from '@internal/core'; +import type { + AppConfiguration, + AppSecrets, + ExtensionDescriptor, + PrismaAppConfig, +} from '@internal/core/config'; import { secretsKeys } from '@internal/core/deploy'; import { CliStructuredError } from '@internal/foundation/errors'; import { readAppConfiguration, readAppSecrets } from './app-configuration.ts'; @@ -202,6 +207,8 @@ export async function runPipeline( }); redaction.addKnownSecrets(valuesUnderSecretsKeys(configuration, secrets, secretsKeys(graph))); + checkGraph(graph, config.extensions); + // 5. Registry coverage: every node/build in the graph has a matching descriptor in the config. validateRegistryCoverage(graph, config); @@ -221,3 +228,16 @@ export async function runPipeline( return { configSource, entryModule, graph, name, assembled, configuration, secrets }; } + +function checkGraph(graph: Graph, extensions: readonly ExtensionDescriptor[]): void { + for (const extension of extensions) { + try { + extension.checkGraph?.(graph); + } catch (error) { + if (error instanceof GraphCheckError) { + throw new GraphCheckError(error.message, { extension: extension.id }); + } + throw error; + } + } +} diff --git a/packages/0-framework/3-tooling/cli/src/redaction.ts b/packages/0-framework/3-tooling/cli/src/redaction.ts index e9e3bbd28..e59aa5f39 100644 --- a/packages/0-framework/3-tooling/cli/src/redaction.ts +++ b/packages/0-framework/3-tooling/cli/src/redaction.ts @@ -1,4 +1,9 @@ -import type { AppConfiguration, AppSecrets } from '@internal/core/config'; +import type { + AppConfiguration, + AppSecrets, + PlatformVariableReader, + UnsetPlatformVariable, +} from '@internal/core/config'; import { CliStructuredError } from '@internal/foundation/errors'; import type { DeploymentSummary } from './deployment-summary.ts'; import type { DevEvent } from './operations/dev.ts'; @@ -87,14 +92,14 @@ export function redactValues(text: string, values: Iterable): string { return new SecretMatcher(values).redact(text); } -/** `text` with bearer and basic tokens, JWTs, the password in a URL, and `token=…`-style pairs replaced by `[redacted]`. */ +/** `text` with bearer and basic tokens, JWTs, the password in a URL, and `token=…`-style pairs replaced by `[redacted]`. The `` placeholder in a printed command is kept. */ function redactShapes(text: string): string { return text .replace(/\b(Bearer|Basic)\s+[\w.~+/=-]+/gi, `$1 ${REDACTED}`) .replace(/\beyJ[\w-]+\.[\w-]+\.[\w-]*/g, REDACTED) .replace(/(\b[a-z][\w+.-]*:\/\/[^\s:/@]*:)[^\s/@]+@/gi, `$1${REDACTED}@`) .replace( - /\b((?:[\w-]*[_-])?(?:token|secret|password|passwd|api[_-]?key|access[_-]?key|private[_-]?key)["']?\s*[:=]\s*["']?)[^\s"',;&]+/gi, + /\b((?:[\w-]*[_-])?(?:token|secret|password|passwd|api[_-]?key|access[_-]?key|private[_-]?key)["']?\s*[:=]\s*["']?)(?!(?:[\s"',;&]|$))[^\s"',;&]+/gi, `$1${REDACTED}`, ); } @@ -215,6 +220,40 @@ export function valuesUnderSecretsKeys( ); } +function redactReader( + reader: PlatformVariableReader, + redaction: SecretRedaction, +): PlatformVariableReader { + switch (reader.kind) { + case 'field': + return { + kind: 'field', + service: redaction.redact(reader.service), + path: redaction.redact(reader.path), + }; + case 'param': + return { + kind: 'param', + service: redaction.redact(reader.service), + param: redaction.redact(reader.param), + }; + case 'unread': + return { kind: 'unread', node: redaction.redact(reader.node), nodeKind: reader.nodeKind }; + } +} + +/** `variable` with every text in it redacted: its name, each reader's names and each command. */ +export function redactUnsetPlatformVariable( + variable: UnsetPlatformVariable, + redaction: SecretRedaction, +): UnsetPlatformVariable { + return { + name: redaction.redact(variable.name), + readers: variable.readers.map((reader) => redactReader(reader, redaction)), + commands: variable.commands.map((command) => redaction.redact(command)), + }; +} + function redactData(value: unknown, redaction: SecretRedaction): unknown { if (typeof value === 'string') return redaction.redact(value); if (Array.isArray(value)) return value.map((member) => redactData(member, redaction)); diff --git a/packages/0-framework/3-tooling/cli/src/render-deployment.ts b/packages/0-framework/3-tooling/cli/src/render-deployment.ts index 79b1761ce..71a18905b 100644 --- a/packages/0-framework/3-tooling/cli/src/render-deployment.ts +++ b/packages/0-framework/3-tooling/cli/src/render-deployment.ts @@ -9,6 +9,7 @@ */ import type { DeployedEntity, DeployedNode, DeploymentResult } from '@internal/core/deploy'; import { writeDeploymentSummaryFile } from './deployment-summary.ts'; +import { unsetPlatformVariablesText } from './unset-platform-variables.ts'; /** Gap between the deepest tree label and the entity column. */ const LABEL_GAP = 3; @@ -115,6 +116,8 @@ export function renderDeployment(result: DeploymentResult): string { } }); } + const unset = unsetPlatformVariablesText(result.findings.unset); + if (unset !== undefined) lines.push('', unset); return lines.join('\n'); } diff --git a/packages/0-framework/3-tooling/cli/src/unset-platform-variables.ts b/packages/0-framework/3-tooling/cli/src/unset-platform-variables.ts new file mode 100644 index 000000000..d21e7cfb6 --- /dev/null +++ b/packages/0-framework/3-tooling/cli/src/unset-platform-variables.ts @@ -0,0 +1,15 @@ +import { platformVariableReaderText, type UnsetPlatformVariable } from '@internal/core/config'; + +const variableLines = (variable: UnsetPlatformVariable): readonly string[] => [ + variable.readers.length === 0 + ? ` - ${variable.name}` + : ` - ${variable.name} (${variable.readers.map(platformVariableReaderText).join('; ')})`, + ...variable.commands.map((command) => ` ${command}`), +]; + +export function unsetPlatformVariablesText( + unset: readonly UnsetPlatformVariable[], +): string | undefined { + if (unset.length === 0) return undefined; + return ['Platform variables with no value:', ...unset.flatMap(variableLines)].join('\n'); +} diff --git a/packages/1-prisma-cloud/0-lowering/input-document/package.json b/packages/1-prisma-cloud/0-lowering/input-document/package.json new file mode 100644 index 000000000..220eada80 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/package.json @@ -0,0 +1,33 @@ +{ + "name": "@internal/input-document", + "version": "0.38.0", + "type": "module", + "exports": { + ".": "./dist/index.mjs", + "./package.json": "./package.json" + }, + "scripts": { + "typecheck": "tsc --noEmit", + "test": "bun test", + "build": "tsdown", + "clean": "rm -rf dist" + }, + "dependencies": {}, + "devDependencies": { + "@internal/tsdown-config": "workspace:0.38.0", + "@types/bun": "^1.3.13", + "tsdown": "^0.22.7", + "typescript": "^6.0.3" + }, + "license": "Apache-2.0", + "repository": { + "type": "git", + "url": "https://github.com/prisma/composer.git", + "directory": "packages/1-prisma-cloud/0-lowering/input-document" + }, + "engines": { + "node": "^22.18.0 || ^24.11.0 || >=26.0.0" + }, + "private": true, + "types": "./dist/index.d.mts" +} diff --git a/packages/1-prisma-cloud/0-lowering/input-document/src/__tests__/document.test.ts b/packages/1-prisma-cloud/0-lowering/input-document/src/__tests__/document.test.ts new file mode 100644 index 000000000..6cb91005e --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/src/__tests__/document.test.ts @@ -0,0 +1,141 @@ +import { describe, expect, test } from 'bun:test'; +import { + collectPlatformMarkers, + describeDocument, + escapeKey, + generatedMarker, + platformMarker, + readDocument, + unescapeKey, + withUnsetVariables, +} from '../exports/index.ts'; + +const boxed = (text: string) => ({ boxed: text }); + +const read = (document: unknown, env: Record) => + readDocument(document, { variable: (name) => env[name], secret: boxed }); + +describe('platformMarker', () => { + test('carries secret: true only for a secret', () => { + expect(platformMarker('X', true)).toEqual({ $platform: { name: 'X', secret: true } }); + expect(platformMarker('X', false)).toEqual({ $platform: { name: 'X' } }); + }); +}); + +describe('the key escape', () => { + test('a user key spelled like a marker gains one "$" and loses it on read', () => { + for (const key of ['$platform', '$$platform', '$generated']) { + expect(escapeKey(key)).toBe(`$${key}`); + expect(unescapeKey(escapeKey(key))).toBe(key); + } + expect(escapeKey('$secret')).toBe('$secret'); + expect(escapeKey('platform')).toBe('platform'); + expect(escapeKey('$platforms')).toBe('$platforms'); + expect(unescapeKey('$platform')).toBe('$platform'); + }); +}); + +describe('collectPlatformMarkers', () => { + test('finds every marker at any depth, with its path, secret flag and array position', () => { + const document = { + apiKey: platformMarker('API_KEY', true), + nested: { $$platform: { from: platformMarker('FROM', false) } }, + list: [platformMarker('LISTED', true)], + token: generatedMarker('COMPOSER_WEB_TOKEN_GENERATED', true), + }; + expect(collectPlatformMarkers(document)).toEqual([ + { name: 'API_KEY', path: 'apiKey', secret: true, inArray: false }, + { name: 'FROM', path: 'nested.$platform.from', secret: false, inArray: false }, + { name: 'LISTED', path: 'list.0', secret: true, inArray: true }, + ]); + }); + + test('a malformed marker is refused, naming its path', () => { + expect(() => collectPlatformMarkers({ a: { $platform: 'X' } })).toThrow( + 'the input document holds a malformed $platform marker at "a"', + ); + }); +}); + +describe('readDocument', () => { + test('a variable with text takes the text, and a secret is boxed', () => { + const document = { from: platformMarker('FROM', false), key: platformMarker('KEY', true) }; + expect(read(document, { FROM: 'shop@example.com', KEY: 'abc' })).toEqual({ + value: { from: 'shop@example.com', key: { boxed: 'abc' } }, + unset: [], + }); + }); + + test('an unset or empty variable leaves its key out, and is named once in unset', () => { + const document = { + a: platformMarker('MISSING', true), + nested: { b: platformMarker('MISSING', true), c: platformMarker('EMPTY', false) }, + }; + expect(read(document, { EMPTY: '' })).toEqual({ + value: { nested: {} }, + unset: ['MISSING', 'EMPTY'], + }); + }); + + test('an unset variable in an array position fails, naming the variable and the path', () => { + expect(() => read({ list: ['a', platformMarker('LISTED', true)] }, {})).toThrow( + 'The input field "list.1" reads the platform variable "LISTED", which has no value. An array position cannot be left out: set "LISTED" with prisma project env add.', + ); + }); + + test('a generated marker is boxed only when redacted, and fails when unset', () => { + const document = { + secret: generatedMarker('G_SECRET', true), + plain: generatedMarker('G_PLAIN', false), + }; + expect(read(document, { G_SECRET: 's', G_PLAIN: 'p' }).value).toEqual({ + secret: { boxed: 's' }, + plain: 'p', + }); + expect(() => read(document, { G_PLAIN: 'p' })).toThrow( + 'The platform variable "G_SECRET", which the deploy writes, has no value. Deploy again.', + ); + }); + + test('escaped user keys are unescaped', () => { + expect(read({ $$platform: 'data', $$$generated: 1 }, {}).value).toEqual({ + $platform: 'data', + $$generated: 1, + }); + }); + + test('a __proto__ key stays an own property and does not set the prototype', () => { + const document = JSON.parse('{"__proto__":{"polluted":true},"nested":{"__proto__":"x"}}'); + const value = read(document, {}).value; + if (typeof value !== 'object' || value === null) throw new Error('expected an object'); + expect(Object.getPrototypeOf(value)).toBe(Object.prototype); + expect(Object.keys(value)).toEqual(['__proto__', 'nested']); + expect(Object.getOwnPropertyDescriptor(value, '__proto__')?.value).toEqual({ polluted: true }); + expect(Object.getOwnPropertyDescriptor(Reflect.get(value, 'nested'), '__proto__')?.value).toBe( + 'x', + ); + }); +}); + +describe('withUnsetVariables', () => { + test('ends the message with the unset names, when there are any', () => { + expect(withUnsetVariables('apiKey must be a string', [])).toBe('apiKey must be a string'); + expect(withUnsetVariables('apiKey must be a string', ['A', 'B'])).toBe( + 'apiKey must be a string (platform variables with no value: A, B)', + ); + }); +}); + +describe('describeDocument', () => { + test('shows each marker as its kind and name, never its JSON', () => { + expect( + describeDocument({ + greeting: 'hello', + key: platformMarker('STRIPE_SECRET_KEY', true), + list: [1, generatedMarker('COMPOSER_WEB_TOKEN_GENERATED', true)], + }), + ).toBe( + '{"greeting":"hello","key":$platform STRIPE_SECRET_KEY,"list":[1,$generated COMPOSER_WEB_TOKEN_GENERATED]}', + ); + }); +}); diff --git a/packages/1-prisma-cloud/0-lowering/input-document/src/document.ts b/packages/1-prisma-cloud/0-lowering/input-document/src/document.ts new file mode 100644 index 000000000..5f3b0271c --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/src/document.ts @@ -0,0 +1,200 @@ +export const PLATFORM_MARKER = '$platform'; +export const GENERATED_MARKER = '$generated'; + +export interface PlatformMarker { + readonly $platform: { readonly name: string; readonly secret?: true }; +} + +export interface GeneratedMarker { + readonly $generated: string; + readonly redacted: boolean; +} + +export interface PlatformMarkerUse { + readonly name: string; + readonly path: string; + readonly secret: boolean; + readonly inArray: boolean; +} + +export const platformMarker = (name: string, secret: boolean): PlatformMarker => ({ + [PLATFORM_MARKER]: secret ? { name, secret: true } : { name }, +}); + +export const generatedMarker = (name: string, redacted: boolean): GeneratedMarker => ({ + [GENERATED_MARKER]: name, + redacted, +}); + +function isPlainObject(value: unknown): value is Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; + const proto = Object.getPrototypeOf(value); + return proto === Object.prototype || proto === null; +} + +export function isPlatformMarker(value: unknown): value is PlatformMarker { + if (!isPlainObject(value)) return false; + const keys = Object.keys(value); + if (keys.length !== 1 || keys[0] !== PLATFORM_MARKER) return false; + const body = value[PLATFORM_MARKER]; + return ( + isPlainObject(body) && + typeof body['name'] === 'string' && + (body['secret'] === undefined || body['secret'] === true) && + Object.keys(body).every((key) => key === 'name' || key === 'secret') + ); +} + +export function isGeneratedMarker(value: unknown): value is GeneratedMarker { + return ( + isPlainObject(value) && + Object.keys(value).length === 2 && + typeof value[GENERATED_MARKER] === 'string' && + typeof value['redacted'] === 'boolean' + ); +} + +const ESCAPABLE_KEY = /^\$+(?:platform|generated)$/; +const ESCAPED_KEY = /^\$\$+(?:platform|generated)$/; + +/** A user key spelled like a marker gains one `$` when written, so only Composer can place a marker. */ +export const escapeKey = (key: string): string => (ESCAPABLE_KEY.test(key) ? `$${key}` : key); + +export const unescapeKey = (key: string): string => (ESCAPED_KEY.test(key) ? key.slice(1) : key); + +const joinPath = (path: string, segment: string | number): string => + path === '' ? String(segment) : `${path}.${segment}`; + +function refuseMalformedMarker(value: Record, path: string): void { + for (const marker of [PLATFORM_MARKER, GENERATED_MARKER]) { + if (marker in value) { + throw new Error( + `the input document holds a malformed ${marker} marker at "${path || '(root)'}". Composer writes every marker; deploy again to rewrite the document.`, + ); + } + } +} + +export function collectPlatformMarkers(document: unknown): PlatformMarkerUse[] { + const uses: PlatformMarkerUse[] = []; + const walk = (value: unknown, path: string, inArray: boolean): void => { + if (isPlatformMarker(value)) { + const { name, secret } = value.$platform; + uses.push({ name, path, secret: secret === true, inArray }); + return; + } + if (isGeneratedMarker(value)) return; + if (Array.isArray(value)) { + value.forEach((member, index) => { + walk(member, joinPath(path, index), true); + }); + return; + } + if (isPlainObject(value)) { + refuseMalformedMarker(value, path); + for (const [key, member] of Object.entries(value)) { + walk(member, joinPath(path, unescapeKey(key)), false); + } + } + }; + walk(document, '', false); + return uses; +} + +export interface ReadDocumentOptions { + readonly variable: (name: string) => string | undefined; + readonly secret: (text: string) => unknown; +} + +export interface ReadDocument { + readonly value: unknown; + /** The platform variables with no value whose keys were left out, each once, in document order. */ + readonly unset: readonly string[]; +} + +const ABSENT: unique symbol = Symbol('input-document-absent'); + +export function readDocument(document: unknown, options: ReadDocumentOptions): ReadDocument { + const unset = new Set(); + const textOf = (name: string): string | undefined => { + const text = options.variable(name); + return text === undefined || text === '' ? undefined : text; + }; + const walk = (value: unknown, path: string, inArray: boolean): unknown => { + if (isPlatformMarker(value)) { + const { name, secret } = value.$platform; + const text = textOf(name); + if (text === undefined) { + if (inArray) throw unsetInArrayError(name, path); + unset.add(name); + return ABSENT; + } + return secret === true ? options.secret(text) : text; + } + if (isGeneratedMarker(value)) { + const name = value.$generated; + const text = textOf(name); + if (text === undefined) { + throw new Error( + `The platform variable "${name}", which the deploy writes, has no value. Deploy again.`, + ); + } + return value.redacted ? options.secret(text) : text; + } + if (Array.isArray(value)) { + return value.map((member, index) => walk(member, joinPath(path, index), true)); + } + if (isPlainObject(value)) { + refuseMalformedMarker(value, path); + const out: Record = {}; + for (const [written, member] of Object.entries(value)) { + const key = unescapeKey(written); + const read = walk(member, joinPath(path, key), false); + if (read !== ABSENT) { + Object.defineProperty(out, key, { + value: read, + enumerable: true, + writable: true, + configurable: true, + }); + } + } + return out; + } + return value; + }; + const value = walk(document, '', false); + return { value: value === ABSENT ? undefined : value, unset: [...unset] }; +} + +function unsetInArrayError(name: string, path: string): Error { + return new Error( + `The input field "${path}" reads the platform variable "${name}", which has no value. An array position cannot be left out: set "${name}" with prisma project env add.`, + ); +} + +export function withUnsetVariables(message: string, unset: readonly string[]): string { + return unset.length === 0 + ? message + : `${message} (platform variables with no value: ${unset.join(', ')})`; +} + +export function requiredParamUnsetMessage(param: string, name: string): string { + return `The param "${param}" of this service reads the platform variable "${name}", which has no value. Set it with prisma project env add.`; +} + +export function describeDocument(document: unknown): string { + const describe = (value: unknown): string => { + if (isPlatformMarker(value)) return `$platform ${value.$platform.name}`; + if (isGeneratedMarker(value)) return `$generated ${value.$generated}`; + if (Array.isArray(value)) return `[${value.map(describe).join(',')}]`; + if (isPlainObject(value)) { + const members = Object.entries(value).map( + ([key, member]) => `${JSON.stringify(key)}:${describe(member)}`, + ); + return `{${members.join(',')}}`; + } + return JSON.stringify(value) ?? 'null'; + }; + return describe(document); +} diff --git a/packages/1-prisma-cloud/0-lowering/input-document/src/env-keys.ts b/packages/1-prisma-cloud/0-lowering/input-document/src/env-keys.ts new file mode 100644 index 000000000..349c0a3fd --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/src/env-keys.ts @@ -0,0 +1,20 @@ +export const addressKeySegments = (address: string): string[] => + address + .split('.') + .filter((s) => s.length > 0) + .map((s) => s.replaceAll('-', '_')); + +export const composerKey = (address: string, ...rest: readonly string[]): string => + ['COMPOSER', ...addressKeySegments(address), ...rest].join('_').toUpperCase(); + +export const ownKeyPrefix = (address: string): string => `${composerKey(address)}_`; + +export const INPUT_KEY_NAME = 'INPUT'; + +const PARAM_POINTER_PREFIX = '@composer-param-pointer:'; + +export const isParamPointerRow = (raw: string): boolean => raw.startsWith(PARAM_POINTER_PREFIX); + +export const encodeParamPointer = (name: string): string => `${PARAM_POINTER_PREFIX}${name}`; + +export const decodeParamPointer = (raw: string): string => raw.slice(PARAM_POINTER_PREFIX.length); diff --git a/packages/1-prisma-cloud/0-lowering/input-document/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/input-document/src/exports/index.ts new file mode 100644 index 000000000..3fb32faf5 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/src/exports/index.ts @@ -0,0 +1,32 @@ +/** + * What the deploy writes and boot reads for a service: the input document and its markers, and the `COMPOSER_` row keys and per-key param pointer rows beside it. + */ +export { + collectPlatformMarkers, + describeDocument, + escapeKey, + GENERATED_MARKER, + type GeneratedMarker, + generatedMarker, + isGeneratedMarker, + isPlatformMarker, + PLATFORM_MARKER, + type PlatformMarker, + type PlatformMarkerUse, + platformMarker, + type ReadDocument, + type ReadDocumentOptions, + readDocument, + requiredParamUnsetMessage, + unescapeKey, + withUnsetVariables, +} from '../document.ts'; +export { + addressKeySegments, + composerKey, + decodeParamPointer, + encodeParamPointer, + INPUT_KEY_NAME, + isParamPointerRow, + ownKeyPrefix, +} from '../env-keys.ts'; diff --git a/packages/1-prisma-cloud/0-lowering/input-document/tsconfig.json b/packages/1-prisma-cloud/0-lowering/input-document/tsconfig.json new file mode 100644 index 000000000..a16ed256c --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/tsconfig.json @@ -0,0 +1,7 @@ +{ + "extends": "../../../../tsconfig.base.json", + "compilerOptions": { + "types": ["bun-types"] + }, + "include": ["src"] +} diff --git a/packages/1-prisma-cloud/0-lowering/input-document/tsdown.config.ts b/packages/1-prisma-cloud/0-lowering/input-document/tsdown.config.ts new file mode 100644 index 000000000..b21e84a8f --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/input-document/tsdown.config.ts @@ -0,0 +1,7 @@ +import { defineConfig } from '@internal/tsdown-config'; + +export default defineConfig({ + entry: { + index: 'src/exports/index.ts', + }, +}); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/package.json b/packages/1-prisma-cloud/0-lowering/local-target/package.json index e58fad1b6..e6b0a1b1e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/package.json +++ b/packages/1-prisma-cloud/0-lowering/local-target/package.json @@ -18,6 +18,7 @@ "@internal/bundle-paths": "workspace:0.38.0", "@internal/core": "workspace:0.38.0", "@internal/dev-emulators": "workspace:0.38.0", + "@internal/input-document": "workspace:0.38.0", "@internal/lowering": "workspace:0.38.0", "@internal/s3-protocol": "workspace:0.38.0", "@prisma/dev": "^0.25.2", diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-materialize-env.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-materialize-env.test.ts index 7772db6c3..5c42141e0 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-materialize-env.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-materialize-env.test.ts @@ -24,10 +24,12 @@ async function writeRows(rows: Record): Promise { test('each service gets the current values of only the variables its own rows point at', async () => { await writeRows({ COMPOSER_BILLING_INPUT: JSON.stringify({ - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, - nested: [{ origin: { $secret: 'APP_ORIGIN' } }], + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, + nested: [{ origin: { $platform: { name: 'APP_ORIGIN', secret: true } } }], + }), + COMPOSER_MAILER_INPUT: JSON.stringify({ + key: { $platform: { name: 'MAILER_KEY', secret: true } }, }), - COMPOSER_MAILER_INPUT: JSON.stringify({ key: { $secret: 'MAILER_KEY' } }), }); const secrets = { STRIPE_SECRET_KEY: 'sk_test_1', @@ -63,7 +65,9 @@ test('a param pointer row gives the service the current value of the variable it test('a value changed in the app secrets reaches the service on the next materialization', async () => { await writeRows({ - COMPOSER_BILLING_INPUT: JSON.stringify({ stripeKey: { $secret: 'STRIPE_SECRET_KEY' } }), + COMPOSER_BILLING_INPUT: JSON.stringify({ + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, + }), }); await materializeEnv(devDir, 'billing', 4000, {}, { STRIPE_SECRET_KEY: 'sk_test_old' }); @@ -81,8 +85,8 @@ test('a value changed in the app secrets reaches the service on the next materia test('a pointed variable that is unset or empty is not given to the service', async () => { await writeRows({ COMPOSER_BILLING_INPUT: JSON.stringify({ - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, - mailer: { $secret: 'MAILER_KEY' }, + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, + mailer: { $platform: { name: 'MAILER_KEY', secret: true } }, }), }); @@ -94,7 +98,9 @@ test('a pointed variable that is unset or empty is not given to the service', as test('a local deployment takes its secrets from the app secrets and its param pointers from the app configuration, never the shell', async () => { await writeRows({ - COMPOSER_BILLING_INPUT: JSON.stringify({ stripeKey: { $secret: 'STRIPE_SECRET_KEY' } }), + COMPOSER_BILLING_INPUT: JSON.stringify({ + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, + }), COMPOSER_BILLING_ORIGIN: '@composer-param-pointer:APP_ORIGIN', }); process.env['STRIPE_SECRET_KEY'] = 'shell'; diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts index 641defd81..f6401a923 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts @@ -14,6 +14,14 @@ import * as path from 'node:path'; import { App, Deployment, EnvironmentVariable, Project } from '@alchemy.run/prisma'; import type { LocalTargetProvidersInput } from '@internal/core/config'; import { computeClient } from '@internal/dev-emulators'; +import { + collectPlatformMarkers, + composerKey, + decodeParamPointer, + INPUT_KEY_NAME, + isParamPointerRow, + ownKeyPrefix, +} from '@internal/input-document'; import * as Provider from 'alchemy/Provider'; import * as Effect from 'effect/Effect'; import type * as Layer from 'effect/Layer'; @@ -67,32 +75,6 @@ async function artifactSha256(artifactPath: string): Promise { return digest; } -/** - * The env-var key the app's own boot-side `deserialize()` reads for its - * `port` service param — `COMPOSER_
_PORT`. Mirrors - * `@prisma/composer-prisma-cloud`'s serializer.ts `configKey(address, { - * owner: 'service', name: 'port' })` byte-for-byte; duplicated rather than - * imported because `@internal/local-target` sits below the extensions layer - * and cannot import a target's serializer (ADR-0028's layer order) — the two - * encode one shared wire protocol (ADR-0029) and must never diverge. - */ -function servicePortEnvKey(address: string): string { - return ['COMPOSER', ...addressKeySegments(address), 'PORT'].join('_').toUpperCase(); -} - -/** The `COMPOSER_
_` prefix every env row this address OWNS carries — `configKey`'s convention (see `servicePortEnvKey`'s doc comment). */ -function ownEnvKeyPrefix(address: string): string { - return `${['COMPOSER', ...addressKeySegments(address)].join('_').toUpperCase()}_`; -} - -/** The serializer's `addressKeySegments`, duplicated for the same reason as `servicePortEnvKey`: `-` in the app name becomes `_`. */ -function addressKeySegments(address: string): string[] { - return address - .split('.') - .filter((s) => s.length > 0) - .map((s) => s.replaceAll('-', '_')); -} - const COMPOSER_NAMESPACE_PREFIX = 'COMPOSER_'; /** @@ -107,7 +89,7 @@ export function scopedEnvRows( allRows: Readonly>, address: string, ): Record { - const ownPrefix = ownEnvKeyPrefix(address); + const ownPrefix = ownKeyPrefix(address); const scoped: Record = {}; for (const [key, value] of Object.entries(allRows)) { if (key.startsWith(ownPrefix) || !key.startsWith(COMPOSER_NAMESPACE_PREFIX)) { @@ -117,40 +99,21 @@ export function scopedEnvRows( return scoped; } -// The pointer formats of `@prisma/composer-prisma-cloud`'s serializer.ts, -// duplicated for the same layer-order reason as `servicePortEnvKey`. -const PARAM_POINTER_PREFIX = '@composer-param-pointer:'; -const SECRET_MARKER = '$secret'; -const INPUT_KEY_NAME = 'INPUT'; - -function collectSecretPointers(value: unknown, out: Set): void { - if (Array.isArray(value)) { - for (const member of value) collectSecretPointers(member, out); - return; - } - if (!Predicate.isObject(value)) return; - const pointer = value[SECRET_MARKER]; - if (Object.keys(value).length === 1 && typeof pointer === 'string') { - out.add(pointer); - return; - } - for (const member of Object.values(value)) collectSecretPointers(member, out); -} - -/** The variables this service's own rows point at: its param pointer rows, and its input document's `$secret` pointers. */ export function pointedVariables( rows: Readonly>, address: string, ): { readonly params: ReadonlySet; readonly secrets: ReadonlySet } { - const ownPrefix = ownEnvKeyPrefix(address); + const ownPrefix = ownKeyPrefix(address); const params = new Set(); const secrets = new Set(); for (const [key, value] of Object.entries(rows)) { if (!key.startsWith(ownPrefix)) continue; - if (value.startsWith(PARAM_POINTER_PREFIX)) { - params.add(value.slice(PARAM_POINTER_PREFIX.length)); + if (isParamPointerRow(value)) { + params.add(decodeParamPointer(value)); } else if (key === `${ownPrefix}${INPUT_KEY_NAME}`) { - collectSecretPointers(JSON.parse(value), secrets); + for (const marker of collectPlatformMarkers(JSON.parse(value))) { + (marker.secret ? secrets : params).add(marker.name); + } } } return { params, secrets }; @@ -222,7 +185,7 @@ export async function materializeEnv( ): Promise> { const rows = await envStore(devDir).read(); const env: Record = scopedEnvRows(rows, address); - env[servicePortEnvKey(address)] = JSON.stringify(port); + env[composerKey(address, 'PORT')] = JSON.stringify(port); const pointed = pointedVariables(rows, address); const setIfText = (name: string, value: unknown): void => { if (typeof value === 'string' && value.length > 0) env[name] = value; diff --git a/packages/1-prisma-cloud/0-lowering/lowering/src/compute/__tests__/deployment-edge.test.ts b/packages/1-prisma-cloud/0-lowering/lowering/src/compute/__tests__/deployment-edge.test.ts index 42676dee0..3b8864e19 100644 --- a/packages/1-prisma-cloud/0-lowering/lowering/src/compute/__tests__/deployment-edge.test.ts +++ b/packages/1-prisma-cloud/0-lowering/lowering/src/compute/__tests__/deployment-edge.test.ts @@ -223,7 +223,9 @@ describe('upstream Deployment.diff while the new variable is still unresolved', */ const envTriggers = (port: string, updatedAt = '2026-01-02T00:00:00.000Z') => ({ COMPOSER_AUTH_PORT: Redacted.make(port), - COMPOSER_AUTH_INPUT: Redacted.make('{"apiKey":{"$secret":"STRIPE_KEY"}}'), + COMPOSER_AUTH_INPUT: Redacted.make( + '{"apiKey":{"$platform":{"name":"STRIPE_KEY","secret":true}}}', + ), 'STRIPE_KEY:updatedAt': updatedAt, }); diff --git a/packages/1-prisma-cloud/1-extensions/target/package.json b/packages/1-prisma-cloud/1-extensions/target/package.json index 20061f13c..b993ea384 100644 --- a/packages/1-prisma-cloud/1-extensions/target/package.json +++ b/packages/1-prisma-cloud/1-extensions/target/package.json @@ -25,6 +25,7 @@ "@internal/dev-emulators": "workspace:0.38.0", "@internal/foundation": "workspace:0.38.0", "@internal/http": "workspace:0.38.0", + "@internal/input-document": "workspace:0.38.0", "@internal/local-target": "workspace:0.38.0", "@internal/lowering": "workspace:0.38.0", "@internal/node": "workspace:0.38.0", diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts index d149a9b0a..7d13a528d 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/bootstrap-service.test.ts @@ -92,7 +92,9 @@ describe('bootstrapService', () => { }, ), ); - expect(rowAtBoot).toBe('{"greeting":"hello","apiKey":{"$secret":"TEST_BOOTSTRAP_API_KEY"}}'); + expect(rowAtBoot).toBe( + '{"greeting":"hello","apiKey":{"$platform":{"name":"TEST_BOOTSTRAP_API_KEY","secret":true}}}', + ); expect(readBack?.greeting).toBe('hello'); expect(readBack?.apiKey.expose()).toBe('sk_test_boot'); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/control-lowering.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/control-lowering.test.ts index f2b8b0d19..166087040 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/control-lowering.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/control-lowering.test.ts @@ -1198,7 +1198,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { }); }); - test('the input document serializes to ONE row — secret leaves are $secret pointers, never values (ADR-0042)', async () => { + test("a service's input is written as one variable that names its secrets and holds none of their values", async () => { await withEnv( // The real secret value is present in the deploy shell, proving it still // cannot reach the serialized row. @@ -1256,7 +1256,8 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { expect(writes).toContainEqual({ project: 'shop-project#cloud-id', key: 'COMPOSER_SHOP_INGEST_INPUT', - value: '{"stripeEnabled":true,"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + value: + '{"stripeEnabled":true,"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', class: 'production', }); // No serialized EnvironmentVariable output carries the secret's value. @@ -1266,7 +1267,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { // variable it points at is named in `pointers`, so its rotation // timestamp can join the triggers — the VALUE is nowhere near it. expect(unwrapTriggers(result.triggers)['COMPOSER_SHOP_INGEST_INPUT']).toBe( - '{"stripeEnabled":true,"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + '{"stripeEnabled":true,"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', ); expect(result.pointers).toEqual(['STRIPE_SECRET_KEY']); expect(JSON.stringify(unwrapTriggers(result.triggers))).not.toContain('sk_live'); @@ -1274,12 +1275,13 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { // put the document (secret-free by construction) on the report entity. expect(result.input).toEqual({ key: 'COMPOSER_SHOP_INGEST_INPUT', - value: '{"stripeEnabled":true,"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + value: + '{"stripeEnabled":true,"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', absent: [], generated: [], // The pointed platform variable, so the deploy hook can fold its // rotation timestamp into the deployment triggers. - secrets: ['STRIPE_SECRET_KEY'], + markers: [{ name: 'STRIPE_SECRET_KEY', path: 'stripeKey', secret: true, inArray: false }], }); }, ); @@ -1543,7 +1545,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { value: '{}', absent: ['greeting → NOT_SET_GREETING_VAR'], generated: [], - secrets: [], + markers: [], }); }); }); @@ -1965,7 +1967,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { port: 3000, input: { key: 'COMPOSER_AUTH_INPUT', - value: '{"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + value: '{"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', absent: ['greeting → ENV_GREETING', 'region → ENV_REGION'], }, }; @@ -1980,7 +1982,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { id: 'auth-svc#cloud-id', url: 'https://shop.auth.deployment.example', details: { - input: '{"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + input: '{"stripeKey":$platform STRIPE_SECRET_KEY}', absent: 'greeting → ENV_GREETING\nregion → ENV_REGION', }, }, @@ -2004,7 +2006,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { port: 3000, input: { key: 'COMPOSER_AUTH_INPUT', - value: '{"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + value: '{"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', absent: [], }, }; @@ -2019,7 +2021,7 @@ describe("prismaCloud().nodes['compute'] — the service descriptor", () => { id: 'auth-svc#cloud-id', url: 'https://shop.auth.deployment.example', details: { - input: '{"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + input: '{"stripeKey":$platform STRIPE_SECRET_KEY}', }, }, ]); @@ -2731,7 +2733,10 @@ describe("descriptors/compute.ts's provider-param loop is generic over the regis const o: ResolvedCloudOptions = { workspaceId: 'ws_1', providerParams, - preflight: { pointerVersion: () => undefined, absentSecrets: new Set() }, + preflight: { + pointerVersion: () => undefined, + absentSecrets: new Set(), + }, }; const node = compute({ name: 'multi', deps: {}, build, expose: { any: anyContract } }); const ctx = { @@ -2845,7 +2850,7 @@ describe('a secret the prisma dev preflight found unset', () => { expect(writes).toContainEqual({ project: 'shop-project#cloud-id', key: 'COMPOSER_SHOP_BILLING_INPUT', - value: '{"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + value: '{"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', class: 'production', }); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/dev-pointed-variables.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/dev-pointed-variables.test.ts index 02ef496c8..93e21ccff 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/dev-pointed-variables.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/dev-pointed-variables.test.ts @@ -1,10 +1,11 @@ -/** The local Deployment provider reads the serializer's pointer formats with its own copy of them; the two must agree. */ import { expect, test } from 'bun:test'; import { service } from '@internal/core'; +import { encodeParamPointer } from '@internal/input-document'; import { pointedVariables } from '@internal/local-target'; import type { StandardSchemaV1 } from '@standard-schema/spec'; +import { configKey } from '../config-key.ts'; import { envSecret } from '../secret.ts'; -import { configKey, encodeParamPointer, serializeInput } from '../serializer.ts'; +import { serializeInput } from '../serializer.ts'; const anySchema: StandardSchemaV1 = { '~standard': { version: 1, vendor: 'test', validate: (value) => ({ value }) }, @@ -25,14 +26,14 @@ const billing = service({ }, }); -test('the local provider finds exactly the variables the serializer writes pointers to', () => { +test('the local provider finds exactly the variables the serializer writes markers for', () => { const input = serializeInput( billing, 'shop.billing', { stripeKey: envSecret('STRIPE_SECRET_KEY'), nested: [{ webhook: envSecret('STRIPE_WEBHOOK_SECRET') }], - userData: { $secret: 'NOT_A_POINTER' }, + userData: { $platform: { name: 'NOT_A_MARKER', secret: true }, $secret: 'OLD_USER_KEY' }, }, { env: {} }, ); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts index da792d349..4cab37ebf 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/extension.test.ts @@ -4,9 +4,10 @@ import { configOf, hydrateSync, isNode, number, param, service, string } from '@ import { secretString } from '@internal/foundation/arktype'; import { RPC_ACCEPTED_KEYS_ENV } from '@internal/service-rpc'; import { type } from 'arktype'; +import { configKey } from '../config-key.ts'; import { compute, rawPostgres, rawPostgresContract } from '../exports/index.ts'; import { bootstrapService } from '../exports/testing.ts'; -import { configKey, deserialize, encode } from '../serializer.ts'; +import { deserialize, encode } from '../serializer.ts'; import { RPC_ACCEPTED_KEYS_PARAM } from '../service-keys.ts'; import { STREAMS_API_KEY_ENV, STREAMS_API_KEY_PARAM } from '../streams-keys.ts'; @@ -789,7 +790,8 @@ describe('the input document — run() re-stash → input() (ADR-0042)', () => { await withEnv( { // deploy-shaped document row (address-keyed) + the user-provisioned platform var: - COMPOSER_INGEST_INPUT: '{"stripeEnabled":true,"stripeKey":{"$secret":"STRIPE_SECRET_KEY"}}', + COMPOSER_INGEST_INPUT: + '{"stripeEnabled":true,"stripeKey":{"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}}}', STRIPE_SECRET_KEY: 'sk_live_trap', COMPOSER_INGEST_PORT: '', // address-free keys the stash (over)writes — tracked so withEnv restores them: diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/graph-check.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/graph-check.test.ts new file mode 100644 index 000000000..b7d675d5c --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/graph-check.test.ts @@ -0,0 +1,98 @@ +import { describe, expect, test } from 'bun:test'; +import { GraphCheckError, Load, module } from '@internal/core'; +import type { StandardSchemaV1 } from '@standard-schema/spec'; +import { type } from 'arktype'; +import { prismaCloud } from '../exports/control.ts'; +import { compute } from '../exports/index.ts'; +import { checkGraph } from '../graph-check.ts'; +import { envParam } from '../param.ts'; +import { envSecret } from '../secret.ts'; + +const build = { + extension: '@prisma/composer/node', + type: 'node', + module: 'file:///test/service.ts', + entry: 'server.js', +}; + +const anySchema: StandardSchemaV1 = { + '~standard': { version: 1, vendor: 'test', validate: (value) => ({ value }) }, +}; + +const web = () => compute({ name: 'web', deps: {}, input: anySchema, build }); + +const thrown = (run: () => void): string => { + try { + run(); + } catch (error) { + if (!(error instanceof GraphCheckError)) throw error; + return error.message; + } + throw new Error('expected the check to throw'); +}; + +describe('the Prisma Cloud graph check', () => { + test('the extension registers it', () => { + expect(prismaCloud().checkGraph).toBe(checkGraph); + }); + + test('refuses a name that envParam() in a binding and a root input field both use, naming both', () => { + const root = module( + 'shop', + { input: type({ APP_ORIGIN: 'string' }) }, + ({ input, provision }) => { + provision(web(), { id: 'site', input: { origin: input.APP_ORIGIN } }); + provision(web(), { id: 'api', input: { links: { origin: envParam('APP_ORIGIN') } } }); + }, + ); + const graph = Load(root, { configuration: { APP_ORIGIN: 'https://shop.example' } }); + + expect(thrown(() => checkGraph(graph))).toBe( + `envParam('APP_ORIGIN') in the input of service "shop.api", field "links.origin" and input.APP_ORIGIN of root module "shop" both use the name "APP_ORIGIN". A name has one meaning in an app: read "APP_ORIGIN" either with envParam() or as a field of the root's input, not both, or give one of them a different name.`, + ); + }); + + test('refuses a name that envParam() in per-key params and a root input field both use', () => { + const root = module( + 'shop', + { input: type({ PORT_NUMBER: 'string' }) }, + ({ input, provision }) => { + provision(web(), { id: 'site', input: { port: input.PORT_NUMBER } }); + provision(compute({ name: 'api', deps: {}, build }), { + id: 'api', + params: { port: envParam('PORT_NUMBER') }, + }); + }, + ); + const graph = Load(root, { configuration: { PORT_NUMBER: '3000' } }); + + expect(thrown(() => checkGraph(graph))).toBe( + `envParam('PORT_NUMBER') in param "port" of service "shop.api" and input.PORT_NUMBER of root module "shop" both use the name "PORT_NUMBER". A name has one meaning in an app: read "PORT_NUMBER" either with envParam() or as a field of the root's input, not both, or give one of them a different name.`, + ); + }); + + test('allows one envParam() name in two bindings and one envSecret() name in two bindings', () => { + const root = module('shop', ({ provision }) => { + provision(web(), { + id: 'site', + input: { origin: envParam('APP_ORIGIN'), key: envSecret('API_KEY') }, + }); + provision(web(), { + id: 'api', + input: { origin: envParam('APP_ORIGIN'), key: envSecret('API_KEY') }, + }); + }); + const graph = Load(root, { configuration: { APP_ORIGIN: 'https://shop.example' } }); + + expect(() => checkGraph(graph)).not.toThrow(); + }); + + test('allows a root input field and an envParam() of different names', () => { + const root = module('shop', { input: type({ FROM: 'string' }) }, ({ input, provision }) => { + provision(web(), { id: 'site', input: { from: input.FROM, origin: envParam('APP_ORIGIN') } }); + }); + const graph = Load(root, { configuration: { FROM: 'shop@example.com', APP_ORIGIN: 'x' } }); + + expect(() => checkGraph(graph)).not.toThrow(); + }); +}); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/input-serializer.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/input-serializer.test.ts index d4b157870..49b0b9832 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/input-serializer.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/input-serializer.test.ts @@ -1,9 +1,3 @@ -/** - * The input channel (ADR-0042): the deploy-side binding walk - * (`resolveInputBinding`), the one-document serialization (`serializeInput`), - * and the boot-side read (`readInput`) — classification, absence, `$secret` - * escaping, validation with pointers in place, and the round trip. - */ import { describe, expect, test } from 'bun:test'; import { isSecretPointer, @@ -107,10 +101,25 @@ describe('resolveInputBinding — the recursive descent (ADR-0042)', () => { expect(absent).toEqual(['unset → NOT_SET_ANYWHERE_XYZ', 'empty → EMPTY_VALUE']); }); - test('an env-bound array element that resolves absent is a loud error — positions cannot be omitted', () => { + test('an array position whose app configuration value is unset fails, sending the user to the app configuration', () => { expect(() => - resolveInputBinding({ list: ['a', envParam('NOT_SET_ANYWHERE_XYZ')] }, '', { env: {} }), - ).toThrow(/array element/); + resolveInputBinding({ list: ['a', envParam('NOT_SET_ANYWHERE_XYZ')] }, 'shop.web', { + env: {}, + }), + ).toThrow( + 'input binding key "list[1]" of service "shop.web" reads "NOT_SET_ANYWHERE_XYZ" from the app configuration, which has no value for it. An array position cannot be left out: set it where a `config` adapter reads it, or bind a literal.', + ); + }); + + test('an array position whose platform variable has no value fails, naming the variable', () => { + expect(() => + resolveInputBinding({ list: [envSecret('UNSET_SECRET')] }, 'shop.web', { + env: {}, + absentSecrets: new Set(['UNSET_SECRET']), + }), + ).toThrow( + 'input binding key "list[0]" of service "shop.web" reads the platform variable "UNSET_SECRET", which has no value. An array position cannot be left out: set the variable, or bind a literal.', + ); }); test('a raw SecretBox (a secret VALUE) in the binding is rejected loudly', () => { @@ -162,11 +171,11 @@ describe('serializeInput — validate with pointers in place, serialize the vali expect(row?.key).toBe('COMPOSER_WEB_INPUT'); expect(JSON.parse(row?.value ?? '')).toEqual({ stripeEnabled: true, - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, }); }); - test('without absent secrets (every deploy), an unset secret is still written as a pointer', () => { + test('the deploy writes a marker for a secret whose variable is unset', () => { const row = serializeInput( svc(conditional), 'web', @@ -175,7 +184,7 @@ describe('serializeInput — validate with pointers in place, serialize the vali ); expect(JSON.parse(row?.value ?? '')).toEqual({ stripeEnabled: true, - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, }); }); @@ -246,7 +255,7 @@ describe('serializeInput — validate with pointers in place, serialize the vali { env: { STRIPE_SECRET_KEY: 'sk_live_should_not_leak' } }, ); expect(row?.value).not.toContain('sk_live'); - expect(row?.value).toContain('"$secret":"STRIPE_SECRET_KEY"'); + expect(row?.value).toContain('"$platform":{"name":"STRIPE_SECRET_KEY","secret":true}'); }); test('a schema that mints its own SecretBox (a default) is rejected — no platform var stands behind it', () => { @@ -281,33 +290,36 @@ describe('serializeInput — validate with pointers in place, serialize the vali }); }); -describe('the reserved $secret key — user data round-trips (ADR-0042)', () => { +describe('user keys spelled like a marker round-trip (ADR-0042)', () => { const anySchema: StandardSchemaV1 = { '~standard': { version: 1, vendor: 'test', validate: (value) => ({ value }) }, }; - test('a user object containing "$secret" (and "$$secret") is escaped on write and unescaped on read', async () => { + test('a user key "$platform" or "$$platform" gains one "$" on write and loses it on read, and "$secret" is written as it is', async () => { const binding = { real: envSecret('REAL_SECRET'), - forged: { $secret: 'NOT_A_POINTER' }, - doubled: { $$secret: 'ALSO_NOT' }, + forged: { $platform: { name: 'NOT_A_MARKER', secret: true } }, + doubled: { $$platform: 'ALSO_NOT' }, + old: { $secret: 'OLD_USER_KEY' }, }; const row = serializeInput(svc(anySchema), 'web', binding, { env: {} }); - // On the wire the user keys carry one extra "$" — only the framework's - // own pointer keeps the single-"$" marker. - expect(row?.value).toContain('"$$secret":"NOT_A_POINTER"'); - expect(row?.value).toContain('"$$$secret":"ALSO_NOT"'); - expect(row?.value).toContain('"real":{"$secret":"REAL_SECRET"}'); + expect(row?.value).toContain('"$$platform":{"name":"NOT_A_MARKER","secret":true}'); + expect(row?.value).toContain('"$$$platform":"ALSO_NOT"'); + expect(row?.value).toContain('"$secret":"OLD_USER_KEY"'); + expect(row?.value).toContain('"real":{"$platform":{"name":"REAL_SECRET","secret":true}}'); + expect(row?.markers).toEqual([ + { name: 'REAL_SECRET', path: 'real', secret: true, inArray: false }, + ]); const node = svc(anySchema); const value = await withEnv( { [inputKey('')]: row?.value ?? '', REAL_SECRET: 'sk_live_real' }, () => readInput(node, '') as Record, ); - // Read back: the pointer became a box; the user keys came back verbatim. expect((value['real'] as SecretBox).expose()).toBe('sk_live_real'); - expect(value['forged']).toEqual({ $secret: 'NOT_A_POINTER' }); - expect(value['doubled']).toEqual({ $$secret: 'ALSO_NOT' }); + expect(value['forged']).toEqual({ $platform: { name: 'NOT_A_MARKER', secret: true } }); + expect(value['doubled']).toEqual({ $$platform: 'ALSO_NOT' }); + expect(value['old']).toEqual({ $secret: 'OLD_USER_KEY' }); }); }); @@ -334,20 +346,91 @@ describe('readInput — the boot half', () => { expect(String(value.stripeKey)).toBe('[REDACTED]'); }); - test('an unprovisioned platform var behind a pointer is a loud boot failure naming both keys', async () => { + test('a required secret whose variable is unset fails the schema, and the error ends with its name', async () => { const node = svc(conditional); await withEnv( - { [inputKey('')]: '{"stripeEnabled":true,"stripeKey":{"$secret":"MISSING_SECRET_VAR"}}' }, + { + [inputKey('')]: + '{"stripeEnabled":true,"stripeKey":{"$platform":{"name":"UNSET_SECRET_VAR","secret":true}}}', + }, + () => { + expect(() => readInput(node, '')).toThrow( + /^The input of service "web" \(environment variable COMPOSER_INPUT\) is invalid: .*\(platform variables with no value: UNSET_SECRET_VAR\)$/, + ); + }, + ); + }); + + test('an optional secret whose variable is unset or empty is left out, and the service boots', async () => { + const node = svc(type({ greeting: 'string', 'apiKey?': secretString() })); + const document = + '{"greeting":"hi","apiKey":{"$platform":{"name":"OPTIONAL_SECRET_VAR","secret":true}}}'; + const unset = await withEnv({ [inputKey('')]: document }, () => readInput(node, '')); + expect(unset).toEqual({ greeting: 'hi' }); + const empty = await withEnv({ [inputKey('')]: document, OPTIONAL_SECRET_VAR: '' }, () => + readInput(node, ''), + ); + expect(empty).toEqual({ greeting: 'hi' }); + }); + + test('a secret whose variable has text is a secret box of that text', async () => { + const node = svc(type({ apiKey: secretString() })); + const value = await withEnv( + { + [inputKey('')]: '{"apiKey":{"$platform":{"name":"PRESENT_SECRET_VAR","secret":true}}}', + PRESENT_SECRET_VAR: 'abc', + }, + () => readInput(node, '') as { apiKey: SecretBox }, + ); + expect(value.apiKey).toBeInstanceOf(SecretBox); + expect(value.apiKey.expose()).toBe('abc'); + }); + + test('a marker in an array position whose variable is unset fails boot naming the variable and the path', async () => { + const node = svc(type({ keys: secretString().array() })); + await withEnv( + { + [inputKey('')]: '{"keys":[{"$platform":{"name":"ARRAY_SECRET_VAR","secret":true}}]}', + }, () => { - expect(() => readInput(node, '')).toThrow(/COMPOSER_INPUT → MISSING_SECRET_VAR/); + expect(() => readInput(node, '')).toThrow( + 'The input field "keys.0" reads the platform variable "ARRAY_SECRET_VAR", which has no value. An array position cannot be left out: set "ARRAY_SECRET_VAR" with prisma project env add.', + ); }, ); }); + test('no part of a secret planted in a value appears in any boot error', async () => { + const planted = 'Zq7Xw9Vb3kLm2Rt'; + const document = + '{"count":"x","apiKey":{"$platform":{"name":"PLANTED_SECRET_VAR","secret":true}},"missing":{"$platform":{"name":"PLANTED_UNSET_VAR","secret":true}}}'; + const messages: string[] = []; + const capture = (schema: StandardSchemaV1) => + withEnv({ [inputKey('')]: document, PLANTED_SECRET_VAR: planted }, () => { + try { + readInput(svc(schema), ''); + } catch (error) { + messages.push((error as Error).message); + } + }); + await capture(type({ count: 'number', apiKey: secretString(), missing: secretString() })); + await capture(type({ count: 'string', apiKey: 'string', missing: secretString() })); + await capture(type({ count: 'string', apiKey: 'number' })); + expect(messages).toHaveLength(3); + for (const message of messages) { + expect(message).toContain('platform variables with no value: PLANTED_UNSET_VAR'); + for (let start = 0; start + 4 <= planted.length; start += 1) { + expect(message).not.toContain(planted.slice(start, start + 4)); + } + } + }); + test('a boot document failing validation is loud, naming the row', async () => { const node = svc(conditional); await withEnv({ [inputKey('')]: '{"stripeEnabled":true}' }, () => { - expect(() => readInput(node, '')).toThrow(/invalid input document \(env COMPOSER_INPUT\)/); + expect(() => readInput(node, '')).toThrow( + /The input of service "web" \(environment variable COMPOSER_INPUT\) is invalid/, + ); }); }); @@ -457,15 +540,16 @@ describe('serializeInput — generated leaves become $generated pointers (ADR-00 expect(value.token).not.toBeInstanceOf(SecretBox); }); - test('a missing generated var at boot is a hard error naming the var and the disagreement', async () => { + test('a missing generated variable at boot fails, naming the variable and telling the user to deploy again', async () => { const node = svc(type({ secret: secretString() })); await withEnv( { [inputKey('')]: '{"secret":{"$generated":"COMPOSER_WEB_SECRET_GENERATED","redacted":true}}', }, () => { - expect(() => readInput(node, '')).toThrow(/COMPOSER_INPUT → COMPOSER_WEB_SECRET_GENERATED/); - expect(() => readInput(node, '')).toThrow(/deploy and the running service disagree/); + expect(() => readInput(node, '')).toThrow( + /"COMPOSER_WEB_SECRET_GENERATED", which the deploy writes, has no value. Deploy again./, + ); }, ); }); @@ -502,38 +586,40 @@ describe('the reserved $generated key — user data round-trips (ADR-0041)', () }); }); -describe('the pointer-name collision guard (ADR-0041, F02)', () => { - test('two secret leaves binding the same platform name collide (secret/secret)', () => { - expect(() => - resolveInputBinding({ a: envSecret('SHARED'), b: envSecret('SHARED') }, 'web', { env: {} }), - ).toThrow(/both resolve to the platform variable "SHARED"/); - }); - - test('two secret leaves whose names differ only in case collide (case-collision)', () => { - expect(() => - resolveInputBinding({ a: envSecret('token'), b: envSecret('TOKEN') }, 'web', { env: {} }), - ).toThrow(/both resolve to the platform variable "TOKEN"/); +describe('distinct names for generated variables only (ADR-0041)', () => { + test('two fields that read the same platform variable are two reads of one variable', () => { + const row = serializeInput( + svc(type({ a: secretString(), b: secretString() })), + 'web', + { a: envSecret('SHARED'), b: envSecret('SHARED') }, + { env: {} }, + ); + expect(JSON.parse(row?.value ?? '')).toEqual({ + a: { $platform: { name: 'SHARED', secret: true } }, + b: { $platform: { name: 'SHARED', secret: true } }, + }); + expect(row?.markers).toEqual([ + { name: 'SHARED', path: 'a', secret: true, inArray: false }, + { name: 'SHARED', path: 'b', secret: true, inArray: false }, + ]); }); test('two generated leaves whose paths normalize to the same var collide (generated/generated)', () => { expect(() => resolveInputBinding({ token: generatedParam(), Token: generatedParam() }, 'web', { env: {} }), - ).toThrow(/both resolve to the platform variable "COMPOSER_WEB_TOKEN_GENERATED"/); + ).toThrow(/both resolve to the generated variable "COMPOSER_WEB_TOKEN_GENERATED"/); }); - test('secret and generated names are checked in one pool (secret-vs-generated)', () => { - // A secret name can never be COMPOSER_-prefixed via envSecret, so this - // cross-namespace collision is exercised on the guard directly: it is one - // general pool, not a per-marker special case. + test('the check names both input paths', () => { expect(() => assertDistinctPointerNames( [ - { path: 'fromSecret', varName: 'COMPOSER_WEB_SECRET_GENERATED' }, - { path: 'fromGenerated', varName: 'COMPOSER_WEB_SECRET_GENERATED' }, + { path: 'first', varName: 'COMPOSER_WEB_SECRET_GENERATED' }, + { path: 'second', varName: 'composer_web_secret_generated' }, ], 'web', ), - ).toThrow(/input paths "fromSecret" and "fromGenerated"/); + ).toThrow(/input paths "first" and "second"/); }); test('a secret leaf and a generated leaf with distinct names do NOT collide', () => { @@ -547,7 +633,7 @@ describe('the pointer-name collision guard (ADR-0041, F02)', () => { }); describe('a secret forwarded through a module input', () => { - test("a secret forwarded through a module's input serializes to its $secret pointer", () => { + test("a secret forwarded through a module's input serializes to its $platform marker", () => { const email = module( 'email', { input: type({ from: 'string', key: secretString() }) }, @@ -569,7 +655,7 @@ describe('a secret forwarded through a module input', () => { expect(JSON.parse(row?.value ?? '')).toEqual({ from: 'shop@example.com', - key: { $secret: 'EMAIL_KEY' }, + key: { $platform: { name: 'EMAIL_KEY', secret: true } }, }); }); @@ -595,7 +681,7 @@ describe('a secret forwarded through a module input', () => { expect(JSON.parse(row?.value ?? '')).toEqual({ user: { $generated: 'COMPOSER_SHOP_EMAIL_WEB_USER_GENERATED', redacted: false }, - password: { $secret: 'SMTP_PASSWORD' }, + password: { $platform: { name: 'SMTP_PASSWORD', secret: true } }, }); }); }); @@ -656,7 +742,7 @@ describe('a source forwarded through a module input', () => { }); describe('an ArkType copy of a source', () => { - test('a copy of envSecret() serializes to its $secret pointer', () => { + test('a copy of envSecret() serializes to its $platform marker', () => { const row = serializeInput( svc(type({ key: secretString() })), 'web', @@ -664,7 +750,9 @@ describe('an ArkType copy of a source', () => { { env: {} }, ); - expect(JSON.parse(row?.value ?? '')).toEqual({ key: { $secret: 'STRIPE_SECRET_KEY' } }); + expect(JSON.parse(row?.value ?? '')).toEqual({ + key: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, + }); }); test('a copy of serviceOrigin() serializes to its $generated pointer', () => { @@ -805,7 +893,7 @@ describe('schemas see pointers at deploy', () => { expect(isSecretString(second?.['token'])).toBe(true); expect(row?.value).not.toContain('composer-stand-in-not-a-secret'); expect(JSON.parse(row?.value ?? '')).toEqual({ - key: { $secret: 'KEY' }, + key: { $platform: { name: 'KEY', secret: true } }, token: { $generated: 'COMPOSER_WEB_TOKEN_GENERATED', redacted: true }, }); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts index 75cdedeae..266d8ecc4 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/invariants.test.ts @@ -148,7 +148,7 @@ describe('invariant 4: environment touches are confined to the config serializer { file: 'control/extension.ts', count: 3 }, { file: 'local-target/attach.ts', count: 1 }, { file: 'preflight.ts', count: 1 }, - { file: 'serializer.ts', count: 9 }, + { file: 'serializer.ts', count: 8 }, { file: 'testing.ts', count: 5 }, ]); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-preflight.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-preflight.test.ts index 638bf26c5..c8d515930 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-preflight.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-preflight.test.ts @@ -3,7 +3,15 @@ import { createHash, createHmac } from 'node:crypto'; import * as fs from 'node:fs'; import * as os from 'node:os'; import * as path from 'node:path'; -import { isSecretPointer, Load, module, resolve, type ServiceNode } from '@internal/core'; +import { + isSecretPointer, + Load, + module, + resolve, + type ServiceNode, + service, + string, +} from '@internal/core'; import { preflightEnv } from '@internal/core/config'; import { isSecretString } from '@internal/foundation/secret'; import { rawHttpContract } from '@internal/http'; @@ -81,6 +89,24 @@ const paramGraph = () => }), ); +/** A service whose per-key param `token` reads API_TOKEN: required unless `token` has a default or is optional. */ +const tokenService = (token = string()) => + service({ + name: 'web', + extension: '@prisma/composer-prisma-cloud', + type: 'compute', + inputs: {}, + params: { token }, + build, + }); + +const tokenGraph = (token = string()) => + Load( + module('app', ({ provision }) => { + provision(tokenService(token), { id: 'web', params: { token: envParam('API_TOKEN') } }); + }), + ); + const mixedGraph = (schema: StandardSchemaV1) => Load( module('app', ({ provision }) => { @@ -88,10 +114,7 @@ const mixedGraph = (schema: StandardSchemaV1) => id: 'billing', input: { stripeKey: envSecret('STRIPE_SECRET_KEY') }, }); - provision(compute({ name: 'web', deps: {}, build }), { - id: 'web', - params: { port: envParam('APP_PORT') }, - }); + provision(tokenService(), { id: 'web', params: { token: envParam('API_TOKEN') } }); }), ); @@ -196,7 +219,7 @@ function billingDocument( ): unknown { const node = graph.nodes.find((entry) => entry.id === 'app.billing')?.node as ServiceNode; const binding = graph.inputBindings.find((b) => b.serviceAddress === 'app.billing')?.binding; - const { absentSecrets } = transportedPreflight(new Map(), childEnv); + const { absentSecrets } = transportedPreflight(() => undefined, childEnv); const row = serializeInput(node, 'app.billing', binding, { env: {}, absentSecrets }); return JSON.parse(row?.value ?? 'null'); } @@ -238,7 +261,7 @@ describe('prisma dev preflight', () => { expect(message).toContain('STRIPE_SECRET_KEY'); expect(message).toContain('service "app.billing"'); - expect(message).toContain('stripeKey is required'); + expect(message).toContain('field "stripeKey"'); expect(message).toContain('the `secrets` list of the `composer` section for a secret'); expect(message).toContain('environmentVariables() reads the shell you run `prisma dev` from'); }); @@ -249,7 +272,9 @@ describe('prisma dev preflight', () => { const message = await preflightError(billingGraph(requiresSecrets('stripeKey'))); - expect(message).toContain('STRIPE_SECRET_KEY (required by service "app.billing"'); + expect(message).toContain( + 'STRIPE_SECRET_KEY (read by service "app.billing", field "stripeKey")', + ); }); test('the error lists every missing required secret and leaves out the optional ones', async () => { @@ -259,14 +284,16 @@ describe('prisma dev preflight', () => { expect(message).not.toContain('STRIPE_WEBHOOK_SECRET'); const both = await preflightError(billingGraph(requiresSecrets('stripeKey', 'webhookSecret'))); - expect(both).toContain('STRIPE_SECRET_KEY (required by service "app.billing"'); - expect(both).toContain('STRIPE_WEBHOOK_SECRET (required by service "app.billing"'); + expect(both).toContain('STRIPE_SECRET_KEY (read by service "app.billing", field "stripeKey")'); + expect(both).toContain( + 'STRIPE_WEBHOOK_SECRET (read by service "app.billing", field "webhookSecret")', + ); }); - test('when the schema needs some of several unset secrets but none alone, the error says so instead of calling each one required', async () => { + test('when the schema needs at least one of several unset secrets but none alone, the error says so instead of calling each one required', async () => { const message = await preflightError(billingGraph(requiresOneOf('stripeKey', 'webhookSecret'))); - expect(message).toContain('some of STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET'); + expect(message).toContain('at least one of STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET'); expect(message).not.toContain('required by service'); }); @@ -277,7 +304,7 @@ describe('prisma dev preflight', () => { const childEnv = transported(await preflight(graph)); expect(billingDocument(graph, childEnv)).toEqual({ - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, }); }); @@ -289,7 +316,7 @@ describe('prisma dev preflight', () => { const childEnv = transported(await preflight(graph)); expect(billingDocument(graph, childEnv)).toEqual({ - stripeKey: { $secret: 'STRIPE_SECRET_KEY' }, + stripeKey: { $platform: { name: 'STRIPE_SECRET_KEY', secret: true } }, }); }); @@ -309,7 +336,7 @@ describe('prisma dev preflight', () => { const memberFor = async (value: string) => { secrets['STRIPE_SECRET_KEY'] = value; const { pointerVersion } = transportedPreflight( - new Map(), + () => undefined, transported(await preflight(graph)), ); return pointerVersion('STRIPE_SECRET_KEY'); @@ -327,23 +354,63 @@ describe('prisma dev preflight', () => { const childEnv = transported(await preflight(billingGraph(requiresSecrets('stripeKey')))); expect( - transportedPreflight(new Map(), childEnv).pointerVersion('STRIPE_WEBHOOK_SECRET'), + transportedPreflight(() => undefined, childEnv).pointerVersion('STRIPE_WEBHOOK_SECRET'), ).toBeUndefined(); }); - test('a missing env-sourced param is a hard error naming the variable and service', async () => { - const message = await preflightError(paramGraph()); + test('a secret an array position reads that is unset stops dev, naming the variable and the service', async () => { + const graph = Load( + module('app', ({ provision }) => { + provision(compute({ name: 'billing', deps: {}, input: requiresSecrets(), build }), { + id: 'billing', + input: { keys: [envSecret('STRIPE_SECRET_KEY')] }, + }); + }), + ); + + const message = await preflightError(graph); + + expect(message).toContain( + '1 platform variable a service needs has no value:\n - STRIPE_SECRET_KEY (read by service "app.billing", field "keys.0")\n required by service "app.billing": an array position reads it, and an array position cannot be left out', + ); + }); + + test('an unset per-key param with no default that is not optional stops dev, naming the variable, the service and the param', async () => { + const message = await preflightError(tokenGraph()); - expect(message).toContain('APP_PORT'); - expect(message).toContain('service "app.web"'); + expect(message).toContain( + '1 platform variable a service needs has no value:\n - API_TOKEN (read by service "app.web", per-key param "token")', + ); expect(message).toContain('the `config` list for an env-sourced param'); }); + test('an empty per-key param with no default that is not optional stops dev the same way', async () => { + configuration['API_TOKEN'] = ''; + + expect(await preflightError(tokenGraph())).toContain('API_TOKEN (read by service "app.web"'); + }); + + test('an unset per-key param with a default does not stop dev and has no fingerprint', async () => { + const payload = await preflight(tokenGraph(string({ default: 'local' }))); + + expect(payload).toBeUndefined(); + }); + + test('an unset per-key param that is optional does not stop dev and has no fingerprint', async () => { + configuration['API_TOKEN'] = ''; + + const payload = await preflight(tokenGraph(string({ optional: true }))); + + expect(payload).toBeUndefined(); + }); + test('missing params and secrets are reported together, with the ways to set them given once', async () => { const message = await preflightError(mixedGraph(requiresSecrets('stripeKey'))); - expect(message).toContain('APP_PORT (required by service "app.web")'); - expect(message).toContain('STRIPE_SECRET_KEY (required by service "app.billing"'); + expect(message).toContain('API_TOKEN (read by service "app.web", per-key param "token")'); + expect(message).toContain( + 'STRIPE_SECRET_KEY (read by service "app.billing", field "stripeKey")', + ); expect(message.split('Set each missing variable').length - 1).toBe(1); }); @@ -353,7 +420,7 @@ describe('prisma dev preflight', () => { expect(message).toContain('invalid input for service "app.billing": somethingElse is required'); expect(message).not.toContain('Deploy-time'); expect(message).not.toContain('STRIPE_SECRET_KEY'); - expect(message).toContain('APP_PORT (required by service "app.web")'); + expect(message).toContain('API_TOKEN (read by service "app.web", per-key param "token")'); }); test('an env-sourced param that is set is fingerprinted for the alchemy process', async () => { @@ -389,11 +456,12 @@ describe('prisma dev preflight', () => { expect(second.pointerVersions.get('APP_PORT')).not.toBe(first.pointerVersions.get('APP_PORT')); }); - test('a secret shared by two services stops dev naming only the service that requires it', async () => { + test('a secret shared by two services stops dev, naming both services that read it', async () => { const message = await preflightError(sharedGraph()); - expect(message).toContain('STRIPE_SECRET_KEY (required by service "app.billing"'); - expect(message).not.toContain('service "app.reports"'); + expect(message).toContain( + 'STRIPE_SECRET_KEY (read by service "app.billing", field "stripeKey"; read by service "app.reports", field "stripeKey")', + ); }); test('a secret shared by two services that is set gets one fingerprint', async () => { @@ -463,7 +531,9 @@ describe('prisma dev preflight', () => { try { const message = await preflightError(billingGraph(requiresSecrets('stripeKey'))); - expect(message).toContain('STRIPE_SECRET_KEY (required by service "app.billing"'); + expect(message).toContain( + 'STRIPE_SECRET_KEY (read by service "app.billing", field "stripeKey")', + ); } finally { delete process.env['STRIPE_SECRET_KEY']; } @@ -487,23 +557,31 @@ describe('prisma dev preflight', () => { }); test('an env-sourced param is read from the app configuration', async () => { - process.env['APP_PORT'] = '3000'; + process.env['API_TOKEN'] = 'from-the-shell'; try { - expect(await preflightError(paramGraph())).toContain( - 'APP_PORT (required by service "app.web")', + expect(await preflightError(tokenGraph())).toContain( + 'API_TOKEN (read by service "app.web", per-key param "token")', ); - configuration['APP_PORT'] = '3000'; - const payload = devPayload(transported(await preflight(paramGraph()))); + configuration['API_TOKEN'] = 'from-the-configuration'; + const payload = devPayload(transported(await preflight(tokenGraph()))); - expect([...payload.pointerVersions.keys()]).toEqual(['APP_PORT']); + expect([...payload.pointerVersions.keys()]).toEqual(['API_TOKEN']); } finally { - delete process.env['APP_PORT']; + delete process.env['API_TOKEN']; } }); + test('a per-key param whose app configuration value is not text stops dev, naming the variable, the service and the kind', async () => { + configuration['API_TOKEN'] = 42; + + expect(await preflightError(tokenGraph())).toBe( + 'The `config` adapters return a number for "API_TOKEN", not text, so prisma dev cannot give service "app.web" the value envParam(\'API_TOKEN\') reads. Make the adapter return a string.', + ); + }); + test('the missing-values message says where the adapters read', async () => { - const message = await preflightError(paramGraph()); + const message = await preflightError(tokenGraph()); expect(message).toContain( 'Set each missing variable where an adapter in prisma.config.ts reads it: the `secrets` list of the `composer` section for a secret, the `config` list for an env-sourced param. environmentVariables() reads the shell you run `prisma dev` from; dotenv() reads .env, .env.local, .env.dev and .env.dev.local beside prisma.config.ts.', diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-serve.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-serve.test.ts index 2d3a63acd..3ebb90ff0 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-serve.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-serve.test.ts @@ -60,7 +60,7 @@ describe.skipIf(process.platform === 'win32')('serving a dev server on an instan fs.writeFileSync( path.join(devDir, 'env.json'), JSON.stringify({ - COMPOSER_GATEWAY_INPUT: '{"apiKey":{"$secret":"STRIPE_KEY"}}', + COMPOSER_GATEWAY_INPUT: '{"apiKey":{"$platform":{"name":"STRIPE_KEY","secret":true}}}', COMPOSER_OTHER_ROW: 'a sibling service row', }), ); @@ -115,7 +115,7 @@ describe.skipIf(process.platform === 'win32')('serving a dev server on an instan PORT: String(instance.port), HOST: '0.0.0.0', COMPOSER_PORT: String(instance.port), - COMPOSER_INPUT: '{"apiKey":{"$secret":"STRIPE_KEY"}}', + COMPOSER_INPUT: '{"apiKey":{"$platform":{"name":"STRIPE_KEY","secret":true}}}', STRIPE_KEY: 'sk_real', // A sibling's row is not this service's to see (scopedEnvRows). COMPOSER_OTHER_ROW: null, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/map-with-concurrency.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/map-with-concurrency.test.ts new file mode 100644 index 000000000..908f9bb66 --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/map-with-concurrency.test.ts @@ -0,0 +1,48 @@ +import { describe, expect, test } from 'bun:test'; +import { mapWithConcurrency } from '../map-with-concurrency.ts'; + +const tick = () => new Promise((resolve) => setTimeout(resolve, 1)); + +describe('mapWithConcurrency', () => { + test('keeps the results in the order of the items, whatever order the tasks finish in', async () => { + const results = await mapWithConcurrency([30, 10, 20], 3, async (ms) => { + await new Promise((resolve) => setTimeout(resolve, ms)); + return ms * 2; + }); + + expect(results).toEqual([60, 20, 40]); + }); + + test('never runs more than the limit at once', async () => { + let running = 0; + let most = 0; + await mapWithConcurrency( + Array.from({ length: 10 }, (_, index) => index), + 3, + async () => { + running += 1; + most = Math.max(most, running); + await tick(); + running -= 1; + }, + ); + + expect(most).toBe(3); + }); + + test('the first failure rejects the call, and no task starts after it', async () => { + const started: number[] = []; + const run = mapWithConcurrency([0, 1, 2, 3, 4], 1, async (item) => { + started.push(item); + await tick(); + if (item === 1) throw new Error('list failed'); + }); + + await expect(run).rejects.toThrow('list failed'); + expect(started).toEqual([0, 1]); + }); + + test('no items gives no results', async () => { + expect(await mapWithConcurrency([], 8, async () => 1)).toEqual([]); + }); +}); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/param.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/param.test.ts index 4582cee9e..73dade92a 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/param.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/param.test.ts @@ -28,11 +28,47 @@ describe('envParam (Prisma Cloud param source)', () => { ); }); - test('rejects empty, COMPOSER_-prefixed, and reserved DATABASE_URL names — parity with envSecret', () => { - expect(() => envParam('')).toThrow(/non-empty/); - expect(() => envParam('COMPOSER_X')).toThrow(/COMPOSER_/); - expect(() => envParam('DATABASE_URL')).toThrow(/reserved/); - expect(() => envParam('DATABASE_URL_POOLED')).toThrow(/reserved/); + test('rejects an empty name', () => { + expect(() => envParam('')).toThrow( + "envParam() requires a non-empty platform variable name, for example envParam('API_KEY').", + ); + }); + + test.each(['api_key', 'Api_Key', '_TOKEN', '1TOKEN', 'API-KEY'])( + 'rejects %s, which is not an upper-case platform variable name', + (name) => { + expect(() => envParam(name)).toThrow( + `envParam name "${name}" is not a platform variable name: it must be an upper-case letter followed by upper-case letters, digits and underscores, for example API_KEY.`, + ); + }, + ); + + test.each([ + ['COMPOSER_X', "names starting with COMPOSER_ are Composer's own platform variables."], + [ + 'DATABASE_URL', + 'Prisma Cloud sets and manages DATABASE_URL and DATABASE_URL_POOLED itself. Declare the database your service uses and read its url from that connection.', + ], + [ + 'DATABASE_URL_POOLED', + 'Prisma Cloud sets and manages DATABASE_URL and DATABASE_URL_POOLED itself. Declare the database your service uses and read its url from that connection.', + ], + ['PORT', "boot sets PORT to the service's port."], + [ + 'PATH', + 'under prisma dev the local target sets PATH and HOME from your shell, so a value on the platform would never be seen locally.', + ], + [ + 'HOME', + 'under prisma dev the local target sets PATH and HOME from your shell, so a value on the platform would never be seen locally.', + ], + ])('rejects the reserved name %s, saying why', (name, reason) => { + expect(() => envParam(name)).toThrow(`envParam name "${name}" is reserved: ${reason}`); + }); + + test('accepts HOST and a name with digits and underscores', () => { + expect(() => envParam('HOST')).not.toThrow(); + expect(() => envParam('STRIPE_KEY_2')).not.toThrow(); }); test('paramName rejects a slot bound to a source not built by envParam', () => { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight-payload.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight-payload.test.ts index 5e68c42a1..235ffd067 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight-payload.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight-payload.test.ts @@ -1,12 +1,3 @@ -/** - * Runs BOTH halves of the rotation-timestamp transport: the CLI half - * produces a payload from a real `runPreflight`, the framework transport - * carries it as env vars, and the alchemy half rebuilds the lookup from - * those vars alone — injecting a lookup directly would pass even if nothing - * were transported. Assertions are on the `:updatedAt` trigger members - * the deploy hook builds from the lookup — the exact inputs - * `Prisma.Deployment.triggers` fingerprints. - */ import { describe, expect, test } from 'bun:test'; import { Load, module } from '@internal/core'; import { preflightEnv } from '@internal/core/config'; @@ -45,28 +36,28 @@ const graph = () => }), ); -/** A platform holding STRIPE_SECRET_KEY, last written at `updatedAt`. Its VALUE is never returned — the API returns none. */ -const fakePlatform = (updatedAt: string): ManagementApiClient => +/** A platform holding STRIPE_SECRET_KEY, last written at `updatedAt`, or no row when `updatedAt` is undefined. Its VALUE is never returned — the API returns none. */ +const fakePlatform = (updatedAt: string | undefined): ManagementApiClient => ({ GET: async () => ({ data: { - data: [{ branchId: null, updatedAt }], + data: updatedAt === undefined ? [] : [{ branchId: null, updatedAt }], pagination: { nextCursor: null, hasMore: false }, }, error: undefined, response: new Response(null, { status: 200 }), }), POST: async () => { - throw new Error('this fake platform already has every name; nothing should be created'); + throw new Error('preflight never creates a platform variable'); }, }) as unknown as ManagementApiClient; /** The pointed platform variables serialize collected for this service — the input document points at the rotating secret. */ const pointers = ['STRIPE_SECRET_KEY']; -/** The CLI process: run the real preflight against a platform that last wrote the secret at `updatedAt`, and hand its findings to the framework transport. */ -async function cliProcess(updatedAt: string): Promise> { - const timestamps = await runPreflight( +/** The real preflight against a platform that last wrote the secret at `updatedAt`. */ +const preflightAgainst = (updatedAt: string | undefined) => + runPreflight( { graph: graph(), configuration: {}, @@ -80,7 +71,10 @@ async function cliProcess(updatedAt: string): Promise> { }, { client: fakePlatform(updatedAt) }, ); - const payload = serializeDeployPreflightPayload(timestamps); + +/** The first process: run the real preflight and hand its payload to the framework transport. */ +async function cliProcess(updatedAt: string | undefined): Promise> { + const payload = serializeDeployPreflightPayload(await preflightAgainst(updatedAt)); return preflightEnv( payload === undefined ? new Map() : new Map([[PRISMA_CLOUD_EXTENSION_ID, payload]]), ); @@ -94,7 +88,7 @@ async function cliProcess(updatedAt: string): Promise> { * fingerprint moves on. */ function alchemyProcessPointerTriggers(env: Record): Record { - const { pointerVersion } = transportedPreflight(new Map(), env); + const { pointerVersion } = transportedPreflight(() => undefined, env); return Object.fromEntries( pointers.map((name) => [`${name}:updatedAt`, pointerVersion(name) ?? '?']), ); @@ -120,10 +114,17 @@ describe('the rotation signal across the CLI → alchemy process boundary', () = expect(env).toEqual({ PRISMA_COMPOSER_PREFLIGHT_PRISMA_COMPOSER_PRISMA_CLOUD: - '{"STRIPE_SECRET_KEY":"2026-05-05T12:00:00.000Z"}', + '{"updatedAt":{"STRIPE_SECRET_KEY":"2026-05-05T12:00:00.000Z"}}', }); }); + test('a variable with no value sends no payload, and its trigger member is unknown', async () => { + const env = await cliProcess(undefined); + + expect(env).toEqual({}); + expect(alchemyProcessPointerTriggers(env)).toEqual({ 'STRIPE_SECRET_KEY:updatedAt': '?' }); + }); + test('without the transport the alchemy process learns nothing — the trigger cannot move', async () => { // What the child saw before the timestamps were transported at all: every // name unknown, so a rotation is invisible. This is the failure the @@ -137,10 +138,12 @@ describe('the rotation signal across the CLI → alchemy process boundary', () = }); describe('what the payload may contain', () => { - test('timestamps only — a name, an ISO time, and nothing else', () => { + test("the payload holds each variable's name and last-change time, and no value", () => { expect( - serializeDeployPreflightPayload(new Map([['STRIPE_SECRET_KEY', '2026-05-05T12:00:00.000Z']])), - ).toBe('{"STRIPE_SECRET_KEY":"2026-05-05T12:00:00.000Z"}'); + serializeDeployPreflightPayload({ + updatedAt: new Map([['STRIPE_SECRET_KEY', '2026-05-05T12:00:00.000Z']]), + }), + ).toBe('{"updatedAt":{"STRIPE_SECRET_KEY":"2026-05-05T12:00:00.000Z"}}'); }); test('the same times in a different order serialize identically — sorted by name', () => { @@ -150,22 +153,25 @@ describe('what the payload may contain', () => { ]); const other = new Map([...one].reverse()); - expect(serializeDeployPreflightPayload(other)).toBe(serializeDeployPreflightPayload(one)); + expect(serializeDeployPreflightPayload({ updatedAt: other })).toBe( + serializeDeployPreflightPayload({ updatedAt: one }), + ); }); - test('a deploy with no pointed variable carries no payload at all', () => { - expect(serializeDeployPreflightPayload(new Map())).toBeUndefined(); + test('a deploy with no variable that has a row carries no payload at all', () => { + expect(serializeDeployPreflightPayload({ updatedAt: new Map() })).toBeUndefined(); }); - test('a round trip preserves every name', () => { - const timestamps = new Map([ + test('a round trip preserves every name and time', () => { + const updatedAt = new Map([ ['A_KEY', '2026-01-01T00:00:00.000Z'], ['B_KEY', '2026-02-02T00:00:00.000Z'], ]); - const payload = serializeDeployPreflightPayload(timestamps); - const decoded = decodePreflightPayload(payload); - expect(decoded.kind === 'deploy' ? [...decoded.updatedAt] : []).toEqual([...timestamps]); + expect(decodePreflightPayload(serializeDeployPreflightPayload({ updatedAt }))).toEqual({ + kind: 'deploy', + updatedAt, + }); }); test('an absent payload decodes as nothing sent', () => { @@ -173,11 +179,29 @@ describe('what the payload may contain', () => { }); test('a payload that is present but unreadable fails loudly rather than losing the signal', () => { - expect(() => decodePreflightPayload('not json')).toThrow(/did not survive the transport/); + expect(() => decodePreflightPayload('not json')).toThrow( + /could not be read in the second process/, + ); expect(() => decodePreflightPayload('"a string"')).toThrow(/not a JSON object/); - expect(() => decodePreflightPayload('{"A_KEY":42}')).toThrow( - /"A_KEY" is not a string timestamp/, + expect(() => decodePreflightPayload('{"updatedAt":{"A_KEY":42}}')).toThrow( + /"updatedAt" is not an object of string timestamps/, + ); + }); +}); + +describe("the CLI process's own preflight", () => { + test('a variable deleted since the last check is treated as having no value', async () => { + let snapshot: ReadonlyMap | undefined; + const { pointerVersion } = transportedPreflight( + () => snapshot, + await cliProcess('2026-05-05T12:00:00.000Z'), ); + + snapshot = (await preflightAgainst('2026-07-07T09:15:00.000Z')).updatedAt; + expect(pointerVersion('STRIPE_SECRET_KEY')).toBe('2026-07-07T09:15:00.000Z'); + + snapshot = (await preflightAgainst(undefined)).updatedAt; + expect(pointerVersion('STRIPE_SECRET_KEY')).toBeUndefined(); }); }); @@ -189,9 +213,11 @@ describe('one transport variable, two kinds of payload', () => { test('a deploy payload names no absent secret and reads as timestamps', () => { const preflight = transportedPreflight( - new Map(), + () => undefined, childEnv( - serializeDeployPreflightPayload(new Map([['STRIPE_SECRET_KEY', '2026-10-01T00:00:00Z']])), + serializeDeployPreflightPayload({ + updatedAt: new Map([['STRIPE_SECRET_KEY', '2026-10-01T00:00:00Z']]), + }), ), ); @@ -201,7 +227,7 @@ describe('one transport variable, two kinds of payload', () => { test('a dev payload carries the absent secrets and the value versions', () => { const preflight = transportedPreflight( - new Map(), + () => undefined, childEnv( serializeDevPreflightPayload({ absentSecrets: ['STRIPE_WEBHOOK_SECRET'], @@ -215,7 +241,7 @@ describe('one transport variable, two kinds of payload', () => { }); test('with no payload nothing is absent and every version is unknown', () => { - const preflight = transportedPreflight(new Map(), childEnv(undefined)); + const preflight = transportedPreflight(() => undefined, childEnv(undefined)); expect([...preflight.absentSecrets]).toEqual([]); expect(preflight.pointerVersion('STRIPE_SECRET_KEY')).toBeUndefined(); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight.test.ts index ab961db66..4bddbd4dc 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/preflight.test.ts @@ -1,7 +1,13 @@ import { beforeEach, describe, expect, test } from 'bun:test'; -import { Load, module } from '@internal/core'; -import type { AppConfiguration, AppSecrets } from '@internal/core/config'; +import { isSecretPointer, Load, module, service, string } from '@internal/core'; +import type { + AppConfiguration, + AppSecrets, + PlatformVariableCheck, + PreflightFindings, +} from '@internal/core/config'; import { secretString } from '@internal/foundation/arktype'; +import { isSecretString } from '@internal/foundation/secret'; import type { ManagementApiClient } from '@internal/lowering'; import type { StandardSchemaV1 } from '@standard-schema/spec'; import { type } from 'arktype'; @@ -37,19 +43,16 @@ interface Row { branchId: string | null; /** Defaulted by the fake client — only the rotation tests below care what it is. */ updatedAt?: string; + isManagedBySystem?: boolean; } /** What a row the test did not date reads as. */ const DEFAULT_UPDATED_AT = '2026-01-01T00:00:00.000Z'; -/** What the fake platform stamps on a row preflight creates from the app configuration or the app secrets. */ -const CREATED_UPDATED_AT = '2026-03-03T00:00:00.000Z'; - interface FakeState { gets: Record[]; posts: Record[]; rows: Row[]; - postStatus: number; /** Page size for the env-var listing — unset serves everything in one page. */ pageSize?: number; /** When set, the listing reports hasMore with a nextCursor equal to the request's cursor — a broken, non-advancing pagination. */ @@ -93,47 +96,73 @@ const fakeClient = (state: FakeState): ManagementApiClient => response: new Response(null, { status: 200 }), }; }, - POST: async (_path: string, init: { body: Record }) => { - state.posts.push(init.body); - if (state.postStatus === 500) { - return { - data: undefined, - error: { code: 'internal', message: 'boom' }, - response: new Response(null, { status: 500 }), - }; - } - if (state.postStatus === 409) { - return { - data: undefined, - error: { code: 'conflict', message: 'already exists' }, - response: new Response(null, { status: 409 }), - }; - } + POST: async (path: string, init: { body: Record }) => { + state.posts.push({ path, ...init.body }); return { - data: { - data: { id: 'ev-new', key: init.body['key'], updatedAt: CREATED_UPDATED_AT }, - }, + data: { data: { id: 'ev-new', key: init.body['key'] } }, error: undefined, response: new Response(null, { status: 201 }), }; }, }) as unknown as ManagementApiClient; -/** Load never validates a binding, so a pass-anything schema is enough here. */ const anySchema: StandardSchemaV1 = { '~standard': { version: 1, vendor: 'test', validate: (value) => ({ value }) }, }; -const secretGraph = () => +/** Passes when every field named in `required` holds a secret: a secret box or a secret pointer. */ +const requiresSecrets = (...required: string[]): StandardSchemaV1 => ({ + '~standard': { + version: 1, + vendor: 'test', + validate: (value) => { + const record = typeof value === 'object' && value !== null ? value : {}; + const missing = required.filter((key) => { + const field = Object.getOwnPropertyDescriptor(record, key)?.value; + return !(isSecretString(field) || isSecretPointer(field)); + }); + return missing.length === 0 + ? { value } + : { issues: missing.map((key) => ({ message: `${key} is required` })) }; + }, + }, +}); + +/** Passes when at least one of `keys` is present. */ +const requiresOneOf = (...keys: string[]): StandardSchemaV1 => ({ + '~standard': { + version: 1, + vendor: 'test', + validate: (value) => { + const record = typeof value === 'object' && value !== null ? value : {}; + return keys.some((key) => key in record) + ? { value } + : { issues: [{ message: `one of ${keys.join(', ')} is required` }] }; + }, + }, +}); + +/** Fails whatever it is given. */ +const neverSchema: StandardSchemaV1 = { + '~standard': { + version: 1, + vendor: 'test', + validate: () => ({ issues: [{ message: 'region is required' }] }), + }, +}; + +const secretGraph = (schema: StandardSchemaV1 = anySchema) => Load( module('app', ({ provision }) => { - provision(compute({ name: 'ingest', deps: {}, input: anySchema, build }), { + provision(compute({ name: 'ingest', deps: {}, input: schema, build }), { id: 'ingest', input: { stripeKey: envSecret('STRIPE_SECRET_KEY') }, }); }), ); +const requiredSecretGraph = () => secretGraph(requiresSecrets('stripeKey')); + const noSecretGraph = () => Load( module('app', ({ provision }) => { @@ -141,7 +170,6 @@ const noSecretGraph = () => }), ); -/** Only a generated leaf — the deploy provisions its var; nothing for an operator to seed. */ const generatedGraph = () => Load( module('app', ({ provision }) => { @@ -152,34 +180,54 @@ const generatedGraph = () => }), ); -/** Two services binding the SAME platform name — the manifest dedups it. */ -const sharedSecretGraph = () => +/** Two services binding the SAME platform name; `web` requires it when `webRequires` is set. */ +const sharedNestedSecretGraph = (webRequires = false) => Load( module('app', ({ provision }) => { - provision(compute({ name: 'web', deps: {}, input: anySchema, build }), { - id: 'web', - // Nested on purpose — preflight's walk must find a leaf at any depth. - input: { stripe: { key: envSecret('STRIPE_SECRET_KEY') } }, - }); + provision( + compute({ + name: 'web', + deps: {}, + input: webRequires ? requiresSecrets('stripe') : anySchema, + build, + }), + { id: 'web', input: { stripe: envSecret('STRIPE_SECRET_KEY') } }, + ); provision(compute({ name: 'ingest', deps: {}, input: anySchema, build }), { id: 'ingest', - input: { key: envSecret('STRIPE_SECRET_KEY') }, + input: { stripe: { key: envSecret('STRIPE_SECRET_KEY') } }, }); }), ); -// The reserved `port` param keeps the env-sourced param channel (ADR-0042): -// binding it to envParam(...) still writes a pointer row the platform must back. -const paramGraph = () => +const defaultedParamGraph = () => Load( module('app', ({ provision }) => { provision(compute({ name: 'web', deps: {}, build }), { id: 'web', - params: { port: envParam('APP_ORIGIN') }, + params: { port: envParam('APP_PORT') }, }); }), ); +/** A per-key param with no default that is not optional: boot fails without its variable. */ +const requiredParamGraph = () => + Load( + module('app', ({ provision }) => { + provision( + service({ + name: 'web', + extension: '@prisma/composer-prisma-cloud', + type: 'compute', + inputs: {}, + params: { token: string() }, + build, + }), + { id: 'web', params: { token: envParam('API_TOKEN') } }, + ); + }), + ); + /** A literal-bound param never touches the platform — preflight must not check it. */ const literalParamGraph = () => Load( @@ -200,10 +248,24 @@ const NO_VALUES: ReadValues = { configuration: {}, secrets: {} }; let read: ReadValues = NO_VALUES; /** Runs preflight with the app configuration and the app secrets `withValues` set, or empty ones. */ -const preflightWith = ( +const preflightWith = async ( input: Omit, deps?: Parameters[1], -) => runPreflight({ ...input, ...read }, deps); +) => { + let findings: PreflightFindings = { unset: [] }; + const payload = await runPreflight( + { + ...input, + ...read, + report: (check) => { + findings = { unset: check.unset }; + input.report?.(check); + }, + }, + deps, + ); + return { ...payload, findings }; +}; /** Sets the app configuration and the app secrets preflight reads for the duration of `fn`. */ async function withValues( @@ -218,286 +280,224 @@ async function withValues( } } -describe('runPreflight — secret manifest verification (ADR-0029)', () => { +const production = { container: fakeContainer('proj', undefined), environment: 'production' }; +const preview = { container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }; + +const failureOf = async (run: Promise): Promise => { + const error = await run.then( + () => undefined, + (e: unknown) => e, + ); + if (!(error instanceof Error)) throw new Error('expected preflight to fail'); + return error.message; +}; + +describe('the deploy check reads which variables exist and sets none', () => { let state: FakeState; beforeEach(() => { - state = { gets: [], posts: [], rows: [], postStatus: 201 }; + state = { gets: [], posts: [], rows: [] }; }); - test('production: checks the production class; all-present passes with no writes', async () => { + test('in production, a variable with a production row passes', async () => { state.rows = [ { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null }, ]; await preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + { graph: requiredSecretGraph(), ...production }, { client: fakeClient(state) }, ); expect(state.gets).toEqual([ { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY' }, ]); - expect(state.posts).toEqual([]); }); - test('named environment: a preview TEMPLATE (branchId null) counts as present', async () => { + test('in a named environment, a preview-wide row counts as a value', async () => { state.rows = [ { projectId: 'proj', class: 'preview', key: 'STRIPE_SECRET_KEY', branchId: null }, ]; await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }, + { graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }, ); expect(state.gets[0]?.['class']).toBe('preview'); - expect(state.posts).toEqual([]); }); - test("named environment: this branch's own OVERRIDE counts as present", async () => { + test("in a named environment, this branch's own row counts as a value", async () => { state.rows = [ { projectId: 'proj', class: 'preview', key: 'STRIPE_SECRET_KEY', branchId: 'br-1' }, ]; await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }, + { graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }, ); - - expect(state.posts).toEqual([]); }); - test("named environment: another branch's override does NOT count — absent from both fails", async () => { + test("in a named environment, another branch's row does not count", async () => { state.rows = [ { projectId: 'proj', class: 'preview', key: 'STRIPE_SECRET_KEY', branchId: 'br-2' }, ]; - const error: unknown = await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }, - { client: fakeClient(state) }, - ).catch((e: unknown) => e); + const message = await failureOf( + preflightWith({ graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }), + ); - expect(error).toBeInstanceOf(Error); - expect((error as Error).message).toContain('STRIPE_SECRET_KEY'); - expect(state.posts).toEqual([]); + expect(message).toContain('STRIPE_SECRET_KEY'); }); - test('fill-missing (named environment): absent-but-in-the-app-secrets is POSTed as a preview branch override', async () => { - state.rows = []; + test('in production, a production row with a branch does not count', async () => { + state.rows = [ + { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: 'br-2' }, + ]; - await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, () => - preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }, - { client: fakeClient(state) }, - ), + const message = await failureOf( + preflightWith({ graph: requiredSecretGraph(), ...production }, { client: fakeClient(state) }), ); - expect(state.posts).toEqual([ - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - value: 'sk_live_fill', - branchId: 'br-1', - }, - ]); + expect(message).toContain('STRIPE_SECRET_KEY'); }); - test('fill-missing (production): POSTed as a production template, no branchId', async () => { - state.rows = []; - - await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + test('sets no platform variable, even when the app secrets hold a value for a missing name', async () => { + await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, async () => { + await preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }); + await preflightWith( + { graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }, - ), - ); + ).catch(() => undefined); + }); + await withValues({ configuration: { APP_PORT: '3100', API_TOKEN: 'tok' } }, async () => { + await preflightWith( + { graph: defaultedParamGraph(), ...production }, + { client: fakeClient(state) }, + ); + await preflightWith( + { graph: requiredParamGraph(), ...production }, + { client: fakeClient(state) }, + ).catch(() => undefined); + }); - expect(state.posts).toEqual([ - { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', value: 'sk_live_fill' }, - ]); + expect(state.gets.length).toBeGreaterThan(0); + expect(state.posts).toEqual([]); }); - test('present on the platform is never overwritten, even when also in the app secrets', async () => { - state.rows = [ - { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null }, - ]; + test('a required variable with no row fails in production, naming the reader and the one command', async () => { + const message = await failureOf( + preflightWith({ graph: requiredSecretGraph(), ...production }, { client: fakeClient(state) }), + ); + + expect(message).toBe( + 'Deploy failed. 1 platform variable a service needs has no value:\n' + + ' - STRIPE_SECRET_KEY (read by service "app.ingest", field "stripeKey")\n\n' + + 'Set it on the platform, then deploy again:\n' + + " - prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role production", + ); + }); - await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_ignored' } }, () => + test('in any other environment the preview-wide command comes first and the branch command second, quoted to paste', async () => { + const message = await failureOf( preflightWith( { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', + graph: requiredSecretGraph(), + container: fakeContainer('proj', 'br_1'), + environment: `it's "$HOME" x`, }, { client: fakeClient(state) }, ), ); - expect(state.posts).toEqual([]); + expect(message).toEndWith( + 'Set it on the platform, then deploy again:\n' + + " - prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role preview\n" + + ` - prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role preview --branch 'it'\\''s "$HOME" x'`, + ); }); - test('absent from both platform and the app secrets fails, naming the missing name, service, and scope', async () => { - state.rows = []; - - const error: unknown = await preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + test('a variable with no row that nothing requires passes and is reported with its reader and commands', async () => { + const result = await preflightWith( + { graph: secretGraph(), ...preview }, { client: fakeClient(state) }, - ).catch((e: unknown) => e); - - expect(error).toBeInstanceOf(Error); - const message = (error as Error).message; - expect(message).toContain('STRIPE_SECRET_KEY'); - expect(message).toContain('service "app.ingest"'); - expect(message).toContain('production'); - expect(message).toContain( - "prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role production", ); - expect(state.posts).toEqual([]); + + expect(result.findings).toEqual({ + unset: [ + { + name: 'STRIPE_SECRET_KEY', + readers: [{ kind: 'field', service: 'app.ingest', path: 'stripeKey' }], + commands: [ + "prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role preview", + "prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --role preview --branch 'pr-1'", + ], + }, + ], + }); + expect(result.updatedAt.size).toBe(0); }); - test('quotes the environment in the printed add command so it is safe to paste', async () => { - state.rows = []; + test('a variable with a row is not a finding', async () => { + state.rows = [ + { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null }, + ]; - const error = await preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', 'br_1'), - environment: `it's "$HOME" x`, - }, + const result = await preflightWith( + { graph: secretGraph(), ...production }, { client: fakeClient(state) }, - ).catch((e: unknown) => e); - - expect(String(error)).toContain( - `prisma project env add 'STRIPE_SECRET_KEY=' --project 'proj' --branch 'it'\\''s "$HOME" x'`, ); + + expect(result.findings).toEqual({ unset: [] }); }); - test('reports how many names it checked, filled from the app configuration and the app secrets, and found missing', async () => { - const reports: Readonly>[] = []; - const run = () => + test('reports how many names it checked with its findings, and nothing when it fails', async () => { + const reports: PlatformVariableCheck[] = []; + const run = (graph: ReturnType) => preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - report: (counts) => reports.push(counts), - }, + { graph, ...production, report: (report) => reports.push(report) }, { client: fakeClient(state) }, ); - await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, run); + const first = await run(secretGraph()); + await run(requiredSecretGraph()).catch(() => undefined); state.rows = [ { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null }, ]; - await run(); - state.rows = []; - await run().catch(() => undefined); + await run(secretGraph()); + expect(first.findings.unset.map((variable) => variable.name)).toEqual(['STRIPE_SECRET_KEY']); expect(reports).toEqual([ - { checked: 1, filled: 1, missing: 0 }, - { checked: 1, filled: 0, missing: 0 }, - { checked: 1, filled: 0, missing: 1 }, + { checked: 1, ...first.findings }, + { checked: 1, unset: [] }, ]); }); - test('a graph with no pointer secrets is a pass-through — no platform calls at all', async () => { - await preflightWith( - { - graph: noSecretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + test('a graph with no platform variable makes no platform call and returns nothing', async () => { + const result = await preflightWith( + { graph: noSecretGraph(), ...production }, { client: fakeClient(state) }, ); expect(state.gets).toEqual([]); - expect(state.posts).toEqual([]); + expect(result).toEqual({ updatedAt: new Map(), findings: { unset: [] } }); }); - test('a generated leaf is skipped — nothing to seed, no platform calls at all', async () => { - await preflightWith( - { - graph: generatedGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ); + test('a value Composer generates needs no platform check', async () => { + await preflightWith({ graph: generatedGraph(), ...production }, { client: fakeClient(state) }); expect(state.gets).toEqual([]); - expect(state.posts).toEqual([]); - }); - - test('a race 409 on fill-missing is tolerated as already-provisioned', async () => { - state.rows = []; - state.postStatus = 409; - - const reports: Readonly>[] = []; - - await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_race' } }, () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - report: (counts) => reports.push(counts), - }, - { client: fakeClient(state) }, - ), - ); - - expect(state.posts).toHaveLength(1); - // The other deploy's row won, so this run filled nothing. - expect(reports).toEqual([{ checked: 1, filled: 0, missing: 0 }]); - }); - - test('a failed fill names the key and the list it came from, never the value', async () => { - state.postStatus = 500; - - const run = withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_SENTINEL' } }, () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - report: () => {}, - }, - { client: fakeClient(state) }, - ), - ); - - await expect(run).rejects.toThrow( - 'deploy preflight: failed to set "STRIPE_SECRET_KEY" on the platform from the app secrets: ' + - '{"code":"internal","message":"boom"}.', - ); }); test('follows pagination — a present name on a later page is not reported missing', async () => { const pages = [ { data: [], pagination: { nextCursor: 'c1', hasMore: true } }, { - data: [ - { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null }, - ], + data: [{ branchId: null, updatedAt: DEFAULT_UPDATED_AT }], pagination: { nextCursor: null, hasMore: false }, }, ]; const queries: Record[] = []; - const posts: unknown[] = []; let call = 0; const client = { GET: async (_path: string, init: { params: { query: Record } }) => { @@ -508,144 +508,273 @@ describe('runPreflight — secret manifest verification (ADR-0029)', () => { response: new Response(null, { status: 200 }), }; }, - POST: async (_path: string, init: { body: Record }) => { - posts.push(init.body); + } as unknown as ManagementApiClient; + + await preflightWith({ graph: requiredSecretGraph(), ...production }, { client }); + + expect(call).toBe(2); + expect(queries[1]?.['cursor']).toBe('c1'); + }); + + test('a failed list call fails the deploy, naming the variable', async () => { + const client = { + GET: async () => ({ + data: undefined, + error: { code: 'internal', message: 'boom' }, + response: new Response(null, { status: 500 }), + }), + } as unknown as ManagementApiClient; + + const message = await failureOf( + preflightWith({ graph: secretGraph(), ...production }, { client }), + ); + + expect(message).toBe( + 'deploy preflight: Prisma Management API error listing "STRIPE_SECRET_KEY": {"code":"internal","message":"boom"}.', + ); + }); + + test('at most eight list calls run at once', async () => { + const names = Array.from({ length: 20 }, (_, index) => `KEY_${index}`); + const graph = Load( + module('app', ({ provision }) => { + provision(compute({ name: 'ingest', deps: {}, input: anySchema, build }), { + id: 'ingest', + input: Object.fromEntries(names.map((name) => [name.toLowerCase(), envSecret(name)])), + }); + }), + ); + let running = 0; + let most = 0; + const listed: string[] = []; + const client = { + GET: async (_path: string, init: { params: { query: Record } }) => { + running += 1; + most = Math.max(most, running); + await new Promise((resolve) => setTimeout(resolve, 2)); + running -= 1; + listed.push(init.params.query['key'] ?? ''); return { - data: { data: { id: 'ev-new', key: init.body['key'] } }, + data: { data: [], pagination: { nextCursor: null, hasMore: false } }, error: undefined, - response: new Response(null, { status: 201 }), + response: new Response(null, { status: 200 }), }; }, } as unknown as ManagementApiClient; - await preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client }, - ); + await preflightWith({ graph, ...production }, { client }); - expect(call).toBe(2); // followed to the second page - expect(queries[1]?.['cursor']).toBe('c1'); // carried the cursor forward - expect(posts).toEqual([]); // found present → no fill + expect(most).toBe(8); + expect(listed.sort()).toEqual([...names].sort()); }); - test('the same platform name bound by two services is checked once and named in the failure', async () => { - state.rows = []; - - const error: unknown = await preflightWith( - { - graph: sharedSecretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + test('the same platform name bound by two services is listed once, and the finding names both readers', async () => { + const result = await preflightWith( + { graph: sharedNestedSecretGraph(), ...production }, { client: fakeClient(state) }, - ).catch((e: unknown) => e); + ); - expect(error).toBeInstanceOf(Error); - expect((error as Error).message).toContain('STRIPE_SECRET_KEY'); - expect((error as Error).message).toMatch(/service "app\.(web|ingest)"/); - // The shared name is deduped to a single platform existence check. expect(state.gets).toHaveLength(1); + expect(result.findings.unset[0]?.readers).toEqual([ + { kind: 'field', service: 'app.web', path: 'stripe' }, + { kind: 'field', service: 'app.ingest', path: 'stripe.key' }, + ]); }); - test('env-sourced param: missing on the platform and absent from the app configuration fails, naming the platform var', async () => { - state.rows = []; + test('a name one of two readers requires fails, and the error names both readers', async () => { + const message = await failureOf( + preflightWith( + { graph: sharedNestedSecretGraph(true), ...production }, + { client: fakeClient(state) }, + ), + ); - const error: unknown = await preflightWith( - { - graph: paramGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ).catch((e: unknown) => e); + expect(message).toContain( + ' - STRIPE_SECRET_KEY (read by service "app.web", field "stripe"; read by service "app.ingest", field "stripe.key")', + ); + }); - expect(error).toBeInstanceOf(Error); - const message = (error as Error).message; - expect(message).toContain('APP_ORIGIN'); - expect(message).toContain('service "app.web"'); - expect(state.posts).toEqual([]); + test('when the schema needs at least one of several unset variables but none alone, the error says so', async () => { + const graph = Load( + module('app', ({ provision }) => { + provision(compute({ name: 'billing', deps: {}, input: requiresOneOf('a', 'b'), build }), { + id: 'billing', + input: { a: envSecret('KEY_A'), b: envSecret('KEY_B') }, + }); + }), + ); + + const message = await failureOf( + preflightWith({ graph, ...production }, { client: fakeClient(state) }), + ); + + expect(message).toStartWith( + 'Deploy failed. 2 platform variables a service needs have no value:\n' + + ' - at least one of KEY_A, KEY_B (service "app.billing" needs at least one of them: one of a, b is required)\n\n' + + 'Set them on the platform, then deploy again:\n', + ); }); - test('env-sourced param: absent on the platform but present in the app configuration is filled from it', async () => { - state.rows = []; + test('only the variables whose absence alone fails the schema are required', async () => { + const graph = Load( + module('app', ({ provision }) => { + provision( + compute({ name: 'billing', deps: {}, input: requiresSecrets('stripeKey'), build }), + { + id: 'billing', + input: { + stripeKey: envSecret('STRIPE_SECRET_KEY'), + webhookSecret: envSecret('STRIPE_WEBHOOK_SECRET'), + }, + }, + ); + }), + ); + + const message = await failureOf( + preflightWith({ graph, ...production }, { client: fakeClient(state) }), + ); + + expect(message).toContain('STRIPE_SECRET_KEY'); + expect(message).not.toContain('STRIPE_WEBHOOK_SECRET'); + }); + + test('a variable in an array position with no row is required, whatever the schema says', async () => { + const graph = Load( + module('app', ({ provision }) => { + provision(compute({ name: 'ingest', deps: {}, input: anySchema, build }), { + id: 'ingest', + input: { keys: [envSecret('KEY_A')] }, + }); + }), + ); - await withValues({ configuration: { APP_ORIGIN: 'https://preview.example.com' } }, () => + const message = await failureOf( + preflightWith({ graph, ...production }, { client: fakeClient(state) }), + ); + + expect(message).toContain( + ' - KEY_A (read by service "app.ingest", field "keys.0")\n' + + ' required by service "app.ingest": an array position reads it, and an array position cannot be left out\n', + ); + }); + + test('an input that fails with every variable present is an invalid input, never blamed on a variable', async () => { + const message = await failureOf( preflightWith( - { - graph: paramGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + { graph: secretGraph(neverSchema), ...production }, { client: fakeClient(state) }, ), ); - expect(state.posts).toEqual([ + expect(message).toStartWith('invalid input for service "app.ingest"'); + expect(message).toContain('region is required'); + expect(message).not.toContain('platform variable a service needs'); + }); + + test('a row the environment sees that the platform manages itself fails, naming the variable', async () => { + state.rows = [ { projectId: 'proj', - class: 'production', - key: 'APP_ORIGIN', - value: 'https://preview.example.com', + class: 'preview', + key: 'STRIPE_SECRET_KEY', + branchId: null, + isManagedBySystem: true, }, - ]); - }); + ]; - test('env-sourced param: present on the platform passes with no writes', async () => { - state.rows = [{ projectId: 'proj', class: 'production', key: 'APP_ORIGIN', branchId: null }]; + const message = await failureOf( + preflightWith({ graph: secretGraph(), ...preview }, { client: fakeClient(state) }), + ); - await preflightWith( + expect(message).toContain( + '"STRIPE_SECRET_KEY" is a platform variable that Prisma Cloud manages itself', + ); + }); + + test("another branch's row that the platform manages is not seen", async () => { + state.rows = [ { - graph: paramGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', + projectId: 'proj', + class: 'preview', + key: 'STRIPE_SECRET_KEY', + branchId: 'br-2', + isManagedBySystem: true, }, + ]; + + await preflightWith({ graph: secretGraph(), ...preview }, { client: fakeClient(state) }); + }); + + test('a per-key envParam() with a default is reported, never required', async () => { + const result = await preflightWith( + { graph: defaultedParamGraph(), ...production }, { client: fakeClient(state) }, ); - expect(state.posts).toEqual([]); + expect(result.findings.unset).toEqual([ + { + name: 'APP_PORT', + readers: [{ kind: 'param', service: 'app.web', param: 'port' }], + commands: ["prisma project env add 'APP_PORT=' --project 'proj' --role production"], + }, + ]); + }); + + test('a per-key envParam() with no default that is not optional is required', async () => { + const message = await failureOf( + preflightWith({ graph: requiredParamGraph(), ...production }, { client: fakeClient(state) }), + ); + + expect(message).toContain( + ' - API_TOKEN (read by service "app.web", per-key param "token")\n' + + ' required by service "app.web": per-key param "token" has no default and is not optional\n', + ); + }); + + test('a per-key envParam() with a row passes', async () => { + state.rows = [{ projectId: 'proj', class: 'production', key: 'API_TOKEN', branchId: null }]; + + await preflightWith( + { graph: requiredParamGraph(), ...production }, + { client: fakeClient(state) }, + ); }); test('a literal-bound param never reaches the platform — no calls at all', async () => { await preflightWith( - { - graph: literalParamGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, + { graph: literalParamGraph(), ...production }, { client: fakeClient(state) }, ); expect(state.gets).toEqual([]); - expect(state.posts).toEqual([]); }); - test("a secret in a module's input that no service receives is checked, named as used by the module", async () => { - const mail = module('mail', { input: type({ 'key?': secretString() }) }, () => {}); + test("a secret in a module's input that no service receives is reported, never required", async () => { + const mail = module('mail', { input: type({ key: secretString() }) }, () => {}); const graph = Load( module('app', ({ provision }) => { provision(mail, { input: { key: envSecret('MAIL_KEY') } }); }), ); - const error: unknown = await preflightWith( - { graph, container: fakeContainer('proj', undefined), environment: 'production' }, - { client: fakeClient(state) }, - ).catch((e: unknown) => e); + const result = await preflightWith({ graph, ...production }, { client: fakeClient(state) }); - expect(error).toBeInstanceOf(Error); - expect((error as Error).message).toContain(' - MAIL_KEY (used by module "app.mail")'); + expect(result.findings.unset).toEqual([ + { + name: 'MAIL_KEY', + readers: [{ kind: 'unread', node: 'app.mail', nodeKind: 'module' }], + commands: ["prisma project env add 'MAIL_KEY=' --project 'proj' --role production"], + }, + ]); }); - test("a secret a service receives through a module's input is named as used by the service", async () => { + test("a secret a service receives through a module's input is named as read by the service", async () => { const mail = module( 'mail', { input: type({ key: secretString() }) }, ({ input, provision }) => { - provision(compute({ name: 'sender', deps: {}, input: anySchema, build }), { + provision(compute({ name: 'sender', deps: {}, input: requiresSecrets('key'), build }), { id: 'sender', input: { key: input.key }, }); @@ -657,13 +786,45 @@ describe('runPreflight — secret manifest verification (ADR-0029)', () => { }), ); - const error: unknown = await preflightWith( - { graph, container: fakeContainer('proj', undefined), environment: 'production' }, - { client: fakeClient(state) }, - ).catch((e: unknown) => e); + const message = await failureOf( + preflightWith({ graph, ...production }, { client: fakeClient(state) }), + ); - expect(error).toBeInstanceOf(Error); - expect((error as Error).message).toContain(' - MAIL_KEY (used by service "app.mail.sender")'); + expect(message).toContain(' - MAIL_KEY (read by service "app.mail.sender", field "key")'); + }); + + test('a value in the app secrets or the shell does not satisfy a required variable', async () => { + process.env['STRIPE_SECRET_KEY'] = 'shell'; + try { + const message = await withValues({ secrets: { STRIPE_SECRET_KEY: 'pool' } }, () => + failureOf( + preflightWith( + { graph: requiredSecretGraph(), ...production }, + { client: fakeClient(state) }, + ), + ), + ); + expect(message).toStartWith( + 'Deploy failed. 1 platform variable a service needs has no value', + ); + } finally { + delete process.env['STRIPE_SECRET_KEY']; + } + }); + + test('no value appears in what preflight hands back or in its error', async () => { + const sentinel = 'sk_live_SENTINEL'; + const result = await withValues({ secrets: { STRIPE_SECRET_KEY: sentinel } }, () => + preflightWith({ graph: secretGraph(), ...preview }, { client: fakeClient(state) }), + ); + const message = await withValues({ secrets: { STRIPE_SECRET_KEY: sentinel } }, () => + failureOf( + preflightWith({ graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }), + ), + ); + + expect(JSON.stringify({ ...result, updatedAt: [...result.updatedAt] })).not.toContain(sentinel); + expect(message).not.toContain(sentinel); }); describe('env-var listing pagination (bounded — drivePagesAsync)', () => { @@ -675,29 +836,19 @@ describe('runPreflight — secret manifest verification (ADR-0029)', () => { ]; await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'pr-1' }, + { graph: requiredSecretGraph(), ...preview }, { client: fakeClient(state) }, ); expect(state.gets).toHaveLength(2); - expect(state.posts).toEqual([]); }); test('a non-advancing cursor fails as broken pagination instead of looping', async () => { - // No matching rows: every page is empty, so the search never - // short-circuits and the stuck cursor is what ends it. state.pageSize = 1; state.cursorStuck = true; await expect( - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), + preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }), ).rejects.toThrow(/pagination appears broken.*possibly incomplete listing/); }); @@ -706,14 +857,7 @@ describe('runPreflight — secret manifest verification (ADR-0029)', () => { state.cursorRunaway = true; await expect( - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), + preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }), ).rejects.toThrow(/did not finish within 1000 pages/); }); @@ -722,187 +866,87 @@ describe('runPreflight — secret manifest verification (ADR-0029)', () => { state.cursorMissing = true; await expect( - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), + preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }), ).rejects.toThrow(/reported more pages but returned no cursor/); }); }); describe('the rotation timestamps it hands back', () => { - test('a name present on the platform reports when it was last written', async () => { - state.rows = [ - { - projectId: 'proj', - class: 'production', - key: 'STRIPE_SECRET_KEY', - branchId: null, - updatedAt: '2026-05-05T12:00:00.000Z', - }, - ]; - - const timestamps = await preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ); - - expect([...timestamps]).toEqual([['STRIPE_SECRET_KEY', '2026-05-05T12:00:00.000Z']]); + const row = (branchId: string | null, updatedAt: string): Row => ({ + projectId: 'proj', + class: 'preview', + key: 'STRIPE_SECRET_KEY', + branchId, + updatedAt, }); + const timestampsFor = async () => + ( + await preflightWith( + { + graph: secretGraph(), + container: fakeContainer('proj', 'br-1'), + environment: 'feature', + }, + { client: fakeClient(state) }, + ) + ).updatedAt; - test('with a template and a branch override in scope, the NEWER row wins', async () => { + test('a name present on the platform reports when it was last written', async () => { state.rows = [ { projectId: 'proj', - class: 'preview', + class: 'production', key: 'STRIPE_SECRET_KEY', branchId: null, updatedAt: '2026-05-05T12:00:00.000Z', }, - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - branchId: 'br-1', - updatedAt: '2026-07-07T12:00:00.000Z', - }, ]; - const timestamps = await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'feature' }, + const { updatedAt } = await preflightWith( + { graph: secretGraph(), ...production }, { client: fakeClient(state) }, ); - expect(timestamps.get('STRIPE_SECRET_KEY')).toBe('2026-07-07T12:00:00.000Z'); + expect([...updatedAt]).toEqual([['STRIPE_SECRET_KEY', '2026-05-05T12:00:00.000Z']]); }); - test('the NEWER row wins even when it is the template, not the branch override', async () => { - state.rows = [ - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - branchId: null, - updatedAt: '2026-08-08T12:00:00.000Z', - }, - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - branchId: 'br-1', - updatedAt: '2026-07-07T12:00:00.000Z', - }, - ]; - - const timestamps = await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'feature' }, - { client: fakeClient(state) }, - ); + test("with a class row and this branch's row, the branch row's time is handed back", async () => { + state.rows = [row(null, '2026-05-05T12:00:00.000Z'), row('br-1', '2026-07-07T12:00:00.000Z')]; - // Recency, not scope precedence: the rotation signal is the newest write - // among every row visible to this environment. - expect(timestamps.get('STRIPE_SECRET_KEY')).toBe('2026-08-08T12:00:00.000Z'); + expect((await timestampsFor()).get('STRIPE_SECRET_KEY')).toBe('2026-07-07T12:00:00.000Z'); }); - test('a row belonging to ANOTHER branch is not in scope and does not date this one', async () => { - state.rows = [ - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - branchId: null, - updatedAt: '2026-05-05T12:00:00.000Z', - }, - { - projectId: 'proj', - class: 'preview', - key: 'STRIPE_SECRET_KEY', - branchId: 'br-other', - updatedAt: '2026-09-09T12:00:00.000Z', - }, - ]; + test('the branch row wins even when the class row is newer', async () => { + state.rows = [row(null, '2026-08-08T12:00:00.000Z'), row('br-1', '2026-07-07T12:00:00.000Z')]; - const timestamps = await preflightWith( - { graph: secretGraph(), container: fakeContainer('proj', 'br-1'), environment: 'feature' }, - { client: fakeClient(state) }, - ); - - expect(timestamps.get('STRIPE_SECRET_KEY')).toBe('2026-05-05T12:00:00.000Z'); + expect((await timestampsFor()).get('STRIPE_SECRET_KEY')).toBe('2026-07-07T12:00:00.000Z'); }); - test('a name preflight fills from the app secrets reports the created row\u2019s time', async () => { - state.rows = []; + test('a newer class row under a branch row changes nothing', async () => { + state.rows = [row(null, '2026-05-05T12:00:00.000Z'), row('br-1', '2026-07-07T12:00:00.000Z')]; + const before = await timestampsFor(); + state.rows = [row(null, '2026-09-09T12:00:00.000Z'), row('br-1', '2026-07-07T12:00:00.000Z')]; - const timestamps = await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), - ); - - expect(timestamps.get('STRIPE_SECRET_KEY')).toBe(CREATED_UPDATED_AT); + expect(await timestampsFor()).toEqual(before); }); - test('a fill that lost the race (409) reports no time, so the next deploy redeploys once', async () => { - state.rows = []; - state.postStatus = 409; + test('with no branch row, the class row is handed back', async () => { + state.rows = [row(null, '2026-05-05T12:00:00.000Z')]; - const timestamps = await withValues({ secrets: { STRIPE_SECRET_KEY: 'sk_live_fill' } }, () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), - ); - - expect(timestamps.has('STRIPE_SECRET_KEY')).toBe(false); + expect((await timestampsFor()).get('STRIPE_SECRET_KEY')).toBe('2026-05-05T12:00:00.000Z'); }); - test('a graph with nothing to check hands back an empty map', async () => { - const timestamps = await preflightWith( - { - graph: noSecretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ); + test('a row belonging to ANOTHER branch is not in scope and does not date this one', async () => { + state.rows = [ + row(null, '2026-05-05T12:00:00.000Z'), + row('br-other', '2026-09-09T12:00:00.000Z'), + ]; - expect(timestamps.size).toBe(0); + expect((await timestampsFor()).get('STRIPE_SECRET_KEY')).toBe('2026-05-05T12:00:00.000Z'); }); - test('no VALUE is ever handed back — the API returns none and preflight asks for none', async () => { - state.rows = []; - - const timestamps = await withValues( - { secrets: { STRIPE_SECRET_KEY: 'sk_live_sentinel' } }, - () => - preflightWith( - { - graph: secretGraph(), - container: fakeContainer('proj', undefined), - environment: 'production', - }, - { client: fakeClient(state) }, - ), - ); - - expect(JSON.stringify([...timestamps])).not.toContain('sk_live_sentinel'); + test('a name with no row has no time', async () => { + expect((await timestampsFor()).has('STRIPE_SECRET_KEY')).toBe(false); }); }); }); @@ -929,7 +973,7 @@ describe('runPreflight — a COMPOSER_ key that changes owner in the move to ful let state: FakeState; beforeEach(() => { - state = { gets: [], posts: [], rows: [], postStatus: 201 }; + state = { gets: [], posts: [], rows: [] }; }); const preflight = () => @@ -1027,7 +1071,6 @@ describe('runPreflight — a chain of COMPOSER_ keys that change owner', () => { gets: [], posts: [], rows: [row('COMPOSER_SHOP_X_ORIGIN'), row('COMPOSER_SHOP_SHOP_X_ORIGIN')], - postStatus: 201, }; const message = String(await preflight(state).catch((e: unknown) => e)); @@ -1045,107 +1088,12 @@ describe('runPreflight — a chain of COMPOSER_ keys that change owner', () => { row('COMPOSER_SHOP_SHOP_X_ORIGIN'), row('COMPOSER_SHOP_SHOP_SHOP_X_ORIGIN'), ], - postStatus: 201, }; await preflight(state); }); }); -describe('runPreflight — values come from the app configuration and the app secrets, never the shell', () => { - let state: FakeState; - - beforeEach(() => { - state = { gets: [], posts: [], rows: [], postStatus: 201 }; - }); - - const production = { container: fakeContainer('proj', undefined), environment: 'production' }; - - /** A name bound both as a secret in the service's input and as an env-sourced param. */ - const sharedNameGraph = () => - Load( - module('app', ({ provision }) => { - provision(compute({ name: 'web', deps: {}, input: anySchema, build }), { - id: 'web', - input: { key: envSecret('SHARED') }, - params: { port: envParam('SHARED') }, - }); - }), - ); - - async function withShell(values: Record, fn: () => Promise): Promise { - for (const [name, value] of Object.entries(values)) process.env[name] = value; - try { - return await fn(); - } finally { - for (const name of Object.keys(values)) delete process.env[name]; - } - } - - test('a missing secret is filled from the app secrets, not the shell', async () => { - await withShell({ STRIPE_SECRET_KEY: 'shell' }, () => - withValues({ secrets: { STRIPE_SECRET_KEY: 'pool' } }, () => - preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }), - ), - ); - - expect(state.posts).toEqual([ - { projectId: 'proj', class: 'production', key: 'STRIPE_SECRET_KEY', value: 'pool' }, - ]); - }); - - test('a missing secret the app secrets lacks fails, even when the shell has it', async () => { - const error = await withShell({ STRIPE_SECRET_KEY: 'shell' }, () => - preflightWith({ graph: secretGraph(), ...production }, { client: fakeClient(state) }).catch( - (e: unknown) => e, - ), - ); - - expect(String(error)).toContain( - 'Deploy failed. 1 required setting has no value:\n - STRIPE_SECRET_KEY (used by service "app.ingest")', - ); - expect(state.posts).toEqual([]); - }); - - test('a missing env-sourced param is filled from the app configuration', async () => { - await withShell({ APP_ORIGIN: 'shell' }, () => - withValues({ configuration: { APP_ORIGIN: 'https://pool.example' } }, () => - preflightWith({ graph: paramGraph(), ...production }, { client: fakeClient(state) }), - ), - ); - - expect(state.posts).toEqual([ - { projectId: 'proj', class: 'production', key: 'APP_ORIGIN', value: 'https://pool.example' }, - ]); - }); - - test('a name bound as a secret and as an env-sourced param is filled from the app secrets', async () => { - await withValues( - { secrets: { SHARED: 'from-secrets' }, configuration: { SHARED: 'from-config' } }, - () => - preflightWith({ graph: sharedNameGraph(), ...production }, { client: fakeClient(state) }), - ); - - expect(state.posts).toEqual([ - { projectId: 'proj', class: 'production', key: 'SHARED', value: 'from-secrets' }, - ]); - }); - - test('an env-sourced param whose app configuration value is not text fails naming its kind', async () => { - const error = await withValues({ configuration: { APP_ORIGIN: 4 } }, () => - preflightWith({ graph: paramGraph(), ...production }, { client: fakeClient(state) }).catch( - (e: unknown) => e, - ), - ); - - expect(error).toBeInstanceOf(Error); - expect((error as Error).message).toBe( - 'The `config` adapters return a number for "APP_ORIGIN", not text, so preflight cannot set the platform variable envParam(\'APP_ORIGIN\') of service "app.web" reads. Make the adapter return a string.', - ); - expect(state.posts).toEqual([]); - }); -}); - describe('preflight names', () => { test('an envSecret() and its ArkType copy are one name to check', () => { const key = envSecret('STRIPE_SECRET_KEY'); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/resource-ids.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/resource-ids.test.ts index 1fd15f89c..c7e2c7484 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/resource-ids.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/resource-ids.test.ts @@ -2,6 +2,7 @@ import { describe, expect, test } from 'bun:test'; import { RenamePolicy } from 'alchemy/Rename'; import * as Effect from 'effect/Effect'; import * as Option from 'effect/Option'; +import { configKey } from '../config-key.ts'; import { addressBelowApp, fromLegacyId, @@ -15,7 +16,6 @@ import { legacyStreamsKeyId, } from '../legacy-resource-ids.ts'; import { resourceId } from '../resource-ids.ts'; -import { configKey } from '../serializer.ts'; describe('resourceId', () => { test("is the address for a node's main resource", () => { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/secret.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/secret.test.ts index db643e559..2bed50d66 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/secret.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/secret.test.ts @@ -17,11 +17,47 @@ describe('envSecret (Prisma Cloud secret source)', () => { ); }); - test('rejects empty, COMPOSER_-prefixed, and reserved DATABASE_URL names', () => { - expect(() => envSecret('')).toThrow(/non-empty/); - expect(() => envSecret('COMPOSER_X')).toThrow(/COMPOSER_/); - expect(() => envSecret('DATABASE_URL')).toThrow(/reserved/); - expect(() => envSecret('DATABASE_URL_POOLED')).toThrow(/reserved/); + test('rejects an empty name', () => { + expect(() => envSecret('')).toThrow( + "envSecret() requires a non-empty platform variable name, for example envSecret('API_KEY').", + ); + }); + + test.each(['api_key', 'Api_Key', '_TOKEN', '1TOKEN', 'API-KEY'])( + 'rejects %s, which is not an upper-case platform variable name', + (name) => { + expect(() => envSecret(name)).toThrow( + `envSecret name "${name}" is not a platform variable name: it must be an upper-case letter followed by upper-case letters, digits and underscores, for example API_KEY.`, + ); + }, + ); + + test.each([ + ['COMPOSER_X', "names starting with COMPOSER_ are Composer's own platform variables."], + [ + 'DATABASE_URL', + 'Prisma Cloud sets and manages DATABASE_URL and DATABASE_URL_POOLED itself. Declare the database your service uses and read its url from that connection.', + ], + [ + 'DATABASE_URL_POOLED', + 'Prisma Cloud sets and manages DATABASE_URL and DATABASE_URL_POOLED itself. Declare the database your service uses and read its url from that connection.', + ], + ['PORT', "boot sets PORT to the service's port."], + [ + 'PATH', + 'under prisma dev the local target sets PATH and HOME from your shell, so a value on the platform would never be seen locally.', + ], + [ + 'HOME', + 'under prisma dev the local target sets PATH and HOME from your shell, so a value on the platform would never be seen locally.', + ], + ])('rejects the reserved name %s, saying why', (name, reason) => { + expect(() => envSecret(name)).toThrow(`envSecret name "${name}" is reserved: ${reason}`); + }); + + test('accepts HOST and a name with digits and underscores', () => { + expect(() => envSecret('HOST')).not.toThrow(); + expect(() => envSecret('STRIPE_KEY_2')).not.toThrow(); }); test('secretName rejects a leaf bound to a source not built by envSecret', () => { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/serializer.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/serializer.test.ts index afcbda30d..2640cc073 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/serializer.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/serializer.test.ts @@ -5,16 +5,18 @@ */ import { beforeEach, describe, expect, test } from 'bun:test'; import { type ConfigParam, number, param, service, string } from '@internal/core'; -import type { StandardSchemaV1 } from '@standard-schema/spec'; import { - configKey, decodeParamPointer, + encodeParamPointer, + isParamPointerRow, +} from '@internal/input-document'; +import type { StandardSchemaV1 } from '@standard-schema/spec'; +import { configKey } from '../config-key.ts'; +import { deserialize, encode, - encodeParamPointer, generatedParamVarName, inputKey, - isParamPointerRow, readOrigin, stashRows, } from '../serializer.ts'; @@ -107,31 +109,64 @@ describe('deserialize — env-sourced param boot resolution', () => { ); }); - test('an unset platform var fails loudly, naming the param and the platform var', async () => { + test('a required param whose platform variable is unset or empty fails boot, saying to set the variable', async () => { + const expected = + 'The param "appOrigin" of this service reads the platform variable "PLATFORM_APP_ORIGIN", which has no value. Set it with prisma project env add.'; await withEnv({ [key]: encodeParamPointer('PLATFORM_APP_ORIGIN') }, () => { - expect(() => deserialize(svc(), address)).toThrow( - /env-sourced config param "appOrigin".*PLATFORM_APP_ORIGIN.*unset/, - ); + expect(() => deserialize(svc(), address)).toThrow(expected); }); - }); - - test('an empty platform var passes through when the schema accepts empty strings', async () => { await withEnv( { [key]: encodeParamPointer('PLATFORM_APP_ORIGIN'), PLATFORM_APP_ORIGIN: '' }, () => { - const config = deserialize(svc(), address); - expect(config.service['appOrigin']).toBe(''); + expect(() => deserialize(svc(), address)).toThrow(expected); }, ); }); - test('an empty platform var fails when the schema rejects empty strings', async () => { - const node = svc({ appOrigin: param(nonEmptyString) }); + test('a param with a default takes the default when its platform variable is unset or empty', async () => { + const node = svc({ appOrigin: string({ default: 'https://default.example' }) }); + for (const value of [undefined, '']) { + await withEnv( + { [key]: encodeParamPointer('PLATFORM_APP_ORIGIN'), PLATFORM_APP_ORIGIN: value }, + () => { + expect(deserialize(node, address).service['appOrigin']).toBe('https://default.example'); + }, + ); + } + }); + + test('an optional param is left out when its platform variable is unset or empty', async () => { + const node = svc({ appOrigin: string({ optional: true }) }); + for (const value of [undefined, '']) { + await withEnv( + { [key]: encodeParamPointer('PLATFORM_APP_ORIGIN'), PLATFORM_APP_ORIGIN: value }, + () => { + expect(Object.hasOwn(deserialize(node, address).service, 'appOrigin')).toBe(false); + }, + ); + } + }); + + test('a platform variable with text the schema rejects fails, whatever the default', async () => { + const node = svc({ appOrigin: param(nonEmptyString, { default: 'x' }) }); + const rejecting = svc({ + appOrigin: param( + { + '~standard': { + version: 1, + vendor: 'test', + validate: () => ({ issues: [{ message: 'must be a url' }] }), + }, + }, + { default: 'x' }, + ), + }); await withEnv( - { [key]: encodeParamPointer('PLATFORM_APP_ORIGIN'), PLATFORM_APP_ORIGIN: '' }, + { [key]: encodeParamPointer('PLATFORM_APP_ORIGIN'), PLATFORM_APP_ORIGIN: 'not a url' }, () => { - expect(() => deserialize(node, address)).toThrow( - /invalid value for env-sourced config param "appOrigin"/, + expect(deserialize(node, address).service['appOrigin']).toBe('not a url'); + expect(() => deserialize(rejecting, address)).toThrow( + /The param "appOrigin" reads the platform variable "PLATFORM_APP_ORIGIN", whose value is invalid/, ); }, ); @@ -202,14 +237,14 @@ describe('stashRows — what run() writes, as a pure function of an environment [configKey(address, { owner: 'service', name: 'port' })]: '3007', [configKey(address, { owner: 'service', name: 'appOrigin' })]: encodeParamPointer('APP_ORIGIN'), APP_ORIGIN: 'https://shop.example', - [inputKey(address)]: '{"apiKey":{"$secret":"STRIPE_KEY"}}', + [inputKey(address)]: '{"apiKey":{"$platform":{"name":"STRIPE_KEY","secret":true}}}', }; test('re-keys every row address-free and adds PORT and the default HOST', () => { expect(stashRows(node(), address, env)).toEqual({ [configKey('', { owner: 'service', name: 'port' })]: '3007', [configKey('', { owner: 'service', name: 'appOrigin' })]: '"https://shop.example"', - [inputKey('')]: '{"apiKey":{"$secret":"STRIPE_KEY"}}', + [inputKey('')]: '{"apiKey":{"$platform":{"name":"STRIPE_KEY","secret":true}}}', PORT: '3007', HOST: '0.0.0.0', }); @@ -230,7 +265,7 @@ describe('stashRows — what run() writes, as a pure function of an environment test("throws run()'s boot error for a row the service cannot read", () => { const { APP_ORIGIN: _unset, ...withoutPlatformVar } = env; expect(() => stashRows(node(), address, withoutPlatformVar)).toThrow( - /env-sourced config param "appOrigin".*APP_ORIGIN.*is unset/, + /The param "appOrigin" of this service reads the platform variable "APP_ORIGIN", which has no value/, ); }); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/compute.ts b/packages/1-prisma-cloud/1-extensions/target/src/compute.ts index 9faa1e985..9fa1866c0 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/compute.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/compute.ts @@ -29,8 +29,9 @@ type ReservedParams = typeof reservedParams; * process-local stash keys, then call boot() to start the app's entry. * · load() / input() — called from inside the app's entry: read the stash; * load() hydrates + memoizes the deps, input() parses the input document, - * replaces each `$secret` pointer with a redacting box over the named - * platform var, validates with the declared schema, and memoizes (ADR-0042). + * replaces each marker with its platform variable's value (a secret in a + * redacting box, an unset one left out), validates with the declared + * schema, and memoizes (ADR-0042). * · origin() — this service's platform-assigned public origin, read from the * stash `run()` populates; memoized per process. * · port() — this service's resolved reserved port (the value `run()` routes diff --git a/packages/1-prisma-cloud/1-extensions/target/src/config-key.ts b/packages/1-prisma-cloud/1-extensions/target/src/config-key.ts index 69f4973c4..b6293e657 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/config-key.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/config-key.ts @@ -1,39 +1,9 @@ -/** - * The env-row key convention the serializer writes and the boot reads, - * kept in a module that imports nothing so the reserved provider params - * (`origin-key.ts`, `streams-keys.ts`) can derive their keys from it at - * module load while `serializer.ts` imports their list - * (`provider-params.ts`) for `stashRows` — without an import cycle. - * `serializer.ts` re-exports both names; import them from there. - */ - -/** - * `COMPOSER_
__`, uppercased. Every - * generated key lives in the framework's reserved COMPOSER_ namespace - * (ADR-0029), so it can never collide with — and silently overwrite — a - * user-provisioned platform var (e.g. a secret's external name). Composer - * writes no unprefixed variable at all: DATABASE_URL(_POOLED) are the - * platform's own names, banned in param.ts/secret.ts and owned by the - * platform (control/extension.ts). - */ -/** - * An address's segments as they appear in a `COMPOSER_` key. The app name may - * contain `-`, which no POSIX variable name may, so it becomes `_` (ADR-0057). - */ -export const addressKeySegments = (address: string): string[] => - address - .split('.') - .filter((s) => s.length > 0) - .map((s) => s.replaceAll('-', '_')); +import { composerKey } from '@internal/input-document'; export const configKey = ( address: string, d: { owner: 'service' | { readonly input: string }; name: string }, -): string => { - const segments = addressKeySegments(address); - const owner = d.owner === 'service' ? [] : [d.owner.input]; - return ['COMPOSER', ...segments, ...owner, d.name].join('_').toUpperCase(); -}; +): string => composerKey(address, ...(d.owner === 'service' ? [] : [d.owner.input]), d.name); /** The framework-resolved origin row: COMPOSER__ORIGIN. Written per * compute service at serialize — the service's own provisioned endpoint URL, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/container.ts b/packages/1-prisma-cloud/1-extensions/target/src/container.ts index 8f1036474..37fe723b7 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/container.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/container.ts @@ -339,7 +339,7 @@ async function removeAppProject( ); const outcome = await runWithClient(program, client); if (outcome.ok) { - console.log(`Removed the Project (${projectId}) — nothing was left in it.`); + console.log(`Removed the Project (${projectId}), with every platform variable set on it.`); return; } if (outcome.error.status === 400) { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/control/extension.ts b/packages/1-prisma-cloud/1-extensions/target/src/control/extension.ts index 3a4b73601..304e4e788 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/control/extension.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/control/extension.ts @@ -32,6 +32,7 @@ import type { ServiceProviderParam, } from '../descriptors/shared.ts'; import { GeneratedParamProvider } from '../generated-param-resource.ts'; +import { checkGraph } from '../graph-check.ts'; import { fromLegacyId, legacyServiceKeyId, legacyStreamsKeyId } from '../legacy-resource-ids.ts'; import { SELF_ORIGIN } from '../origin-key.ts'; import { OrmMigrationProvider } from '../orm-migration-resource.ts'; @@ -334,14 +335,17 @@ export const prismaCloud = (opts: PrismaCloudOptions = {}): ExtensionDescriptor // module variable, so two `prismaCloud()` extensions cannot share it. In // the alchemy process (no preflight) the lookup falls back to what the CLI // transported (preflight-payload.ts). - const preflightTimestamps = new Map(); - const o = lazyOptions(opts, transportedPreflight(preflightTimestamps, process.env)); + let preflightTimestamps: ReadonlyMap | undefined; + const transported = transportedPreflight(() => preflightTimestamps, process.env); + const o = lazyOptions(opts, transported); return { id: PRISMA_CLOUD_EXTENSION_ID, container: containerDescriptor({ region: () => o().region }), + checkGraph, + providers: () => asProvidersLayer( Layer.mergeAll( @@ -354,17 +358,10 @@ export const prismaCloud = (opts: PrismaCloudOptions = {}): ExtensionDescriptor ), ), - // Deploy-time prerequisite check (ADR-0029): verify every pointer secret in - // the provision manifest exists for the resolved environment, filling absent-but- - // in-shell names via a direct API POST — before any stack file or Alchemy. - // Timestamps are kept for this process AND serialized onto the preflight - // transport for the alchemy process. The parameter annotation is what - // recovers this extension's own client type from the framework's erased - // one; without it `input.credentials` arrives as `unknown`. preflight: (input: PrismaCloudPreflightInput) => - runPreflight(input).then((timestamps) => { - for (const [name, updatedAt] of timestamps) preflightTimestamps.set(name, updatedAt); - return serializeDeployPreflightPayload(timestamps); + runPreflight(input).then((result) => { + preflightTimestamps = result.updatedAt; + return serializeDeployPreflightPayload(result); }), // Records the deploy as a Build so it appears in the Console, and passes diff --git a/packages/1-prisma-cloud/1-extensions/target/src/control/preflight-payload.ts b/packages/1-prisma-cloud/1-extensions/target/src/control/preflight-payload.ts index 1072130a7..f11d00189 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/control/preflight-payload.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/control/preflight-payload.ts @@ -1,10 +1,9 @@ /** * What this extension's preflight hands the alchemy process, which re-imports - * the config from scratch, on the framework's preflight transport. One - * variable carries either kind: + * the config from scratch, in one environment variable that carries either kind: * - * - deploy: a JSON object mapping each pointed platform variable to its - * `updatedAt`, as read from the Management API; + * - deploy: a JSON object holding `updatedAt`, which maps each platform + * variable a service reads to the `updatedAt` of the row the platform resolves; * - `prisma dev`: `dev:` followed by JSON naming the unset secrets and an * HMAC-SHA256 of the current value of each variable a service points at. * @@ -24,6 +23,10 @@ export interface TransportedPreflight { readonly absentSecrets: ReadonlySet; } +export interface DeployPreflightPayload { + readonly updatedAt: ReadonlyMap; +} + export interface DevPreflightPayload { readonly absentSecrets: readonly string[]; readonly pointerVersions: Readonly>; @@ -31,7 +34,7 @@ export interface DevPreflightPayload { export type DecodedPreflightPayload = | { readonly kind: 'none' } - | { readonly kind: 'deploy'; readonly updatedAt: ReadonlyMap } + | ({ readonly kind: 'deploy' } & DeployPreflightPayload) | { readonly kind: 'dev'; readonly absentSecrets: ReadonlySet; @@ -42,12 +45,11 @@ const DEV_PAYLOAD_PREFIX = 'dev:'; const byName = ([a]: [string, unknown], [b]: [string, unknown]) => (a < b ? -1 : 1); -/** The deploy preflight's payload, or undefined when it read no pointed variable. */ export function serializeDeployPreflightPayload( - updatedAt: ReadonlyMap, + payload: DeployPreflightPayload, ): string | undefined { - if (updatedAt.size === 0) return undefined; - return JSON.stringify(Object.fromEntries([...updatedAt].sort(byName))); + if (payload.updatedAt.size === 0) return undefined; + return JSON.stringify({ updatedAt: Object.fromEntries([...payload.updatedAt].sort(byName)) }); } export function serializeDevPreflightPayload(payload: DevPreflightPayload): string { @@ -100,34 +102,26 @@ export function decodePreflightPayload(payload: string | undefined): DecodedPref if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) { throw deployPayloadError('it is not a JSON object'); } - const updatedAt = new Map(); - for (const [name, value] of Object.entries(parsed)) { - if (typeof value !== 'string') throw deployPayloadError(`"${name}" is not a string timestamp`); - updatedAt.set(name, value); + if (!('updatedAt' in parsed) || !isStringRecord(parsed.updatedAt)) { + throw deployPayloadError('its "updatedAt" is not an object of string timestamps'); } - return { kind: 'deploy', updatedAt }; + return { kind: 'deploy', updatedAt: new Map(Object.entries(parsed.updatedAt)) }; } const deployPayloadError = (reason: string): Error => new Error( - "prisma-cloud: the deploy preflight's rotation timestamps did not survive the transport " + - `into the alchemy process — ${reason}. This is a framework bug; re-running the deploy ` + - 'will not fix it.', + `prisma-cloud: the deploy check's payload could not be read in the second process: ${reason}. ` + + 'This is a Composer bug; deploying again will not fix it.', ); const devPayloadError = (reason: string): Error => new Error( - "prisma-cloud: the `prisma dev` preflight's payload did not survive the transport into the " + - `alchemy process — ${reason}. This is a framework bug; re-running \`prisma dev\` will not fix it.`, + `prisma-cloud: the \`prisma dev\` check's payload could not be read in the second process: ${reason}. ` + + 'This is a Composer bug; running `prisma dev` again will not fix it.', ); -/** - * The lowering's view of what the preflight sent. `own` holds what the deploy - * preflight read in this same process (the CLI process); the alchemy process - * has none and reads the transported payload, decoded once on first use. - */ export function transportedPreflight( - own: ReadonlyMap, + own: () => ReadonlyMap | undefined, env: Readonly>, ): TransportedPreflight { let decoded: DecodedPreflightPayload | undefined; @@ -136,8 +130,8 @@ export function transportedPreflight( const noSecrets: ReadonlySet = new Set(); return { pointerVersion: (name) => { - const mine = own.get(name); - if (mine !== undefined) return mine; + const snapshot = own(); + if (snapshot !== undefined) return snapshot.get(name); const payload = decode(); if (payload.kind === 'deploy') return payload.updatedAt.get(name); if (payload.kind === 'dev') return payload.pointerVersions.get(name); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/descriptors/compute.ts b/packages/1-prisma-cloud/1-extensions/target/src/descriptors/compute.ts index 3b408e92d..f9b14dadf 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/descriptors/compute.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/descriptors/compute.ts @@ -3,6 +3,13 @@ import * as Prisma from '@alchemy.run/prisma'; import { isParamSource } from '@internal/core'; import type { ServiceLowering } from '@internal/core/deploy'; +import { + decodeParamPointer, + describeDocument, + encodeParamPointer, + isParamPointerRow, + type PlatformMarkerUse, +} from '@internal/input-document'; import { ARTIFACT_CONTENT_TYPE, appAfterEnvironment, @@ -11,6 +18,7 @@ import { import * as Output from 'alchemy/Output'; import * as Effect from 'effect/Effect'; import * as Redacted from 'effect/Redacted'; +import { configKey } from '../config-key.ts'; import { GeneratedParam } from '../generated-param-resource.ts'; import { fromLegacyId, @@ -24,16 +32,7 @@ import { import { paramBindingFor, paramName } from '../param.ts'; import { provisionedEdges } from '../provisioned-edges.ts'; import { resourceId } from '../resource-ids.ts'; -import { - configKey, - decodeParamPointer, - encode, - encodeParamPointer, - type InputDocumentRow, - isParamPointerRow, - paramEntries, - serializeInput, -} from '../serializer.ts'; +import { encode, type InputDocumentRow, paramEntries, serializeInput } from '../serializer.ts'; import { asOwnRow, cloudApplicationOf, @@ -79,17 +78,10 @@ export interface ComputeSerialized { * replacement. Alchemy also writes the members to its state as given, * `Redacted` values in plain text included. Because upstream resolves the actual * values, a value re-issued under a stable resource identity (a connection - * rotated in place, a re-minted service key) moves the fingerprint too — - * closing the blind spot the deleted Composer-side fingerprint had to accept - * (ADR-0048 recorded it). + * rotated in place, a re-minted service key) moves the fingerprint too. */ readonly triggers: Readonly>; - /** - * Operator-owned platform variables the environment POINTS at and Composer - * never writes — no value ever exists to trigger on, so each contributes its - * preflight `updatedAt` as a `:updatedAt` trigger member at deploy - * time, and an out-of-band rotation still replaces the deployment. - */ + /** Platform variables the service reads and Composer does not write; each joins the triggers as `:updatedAt`. */ readonly pointers: readonly string[]; readonly port: number; readonly input?: InputDocumentRow; @@ -147,20 +139,16 @@ interface EnvRow { readonly pointers?: readonly string[]; } -/** - * The report's two lines of honesty (ADR-0042): the serialized input - * document (secret-free by construction — every secret leaf is a `$secret` - * pointer) and every binding key that resolved absent in the app configuration (a - * possible typo'd variable name). Newlines in a detail value render as one - * line per entry. - */ +const sortedMarkerNames = (markers: readonly PlatformMarkerUse[]): string[] => + [...new Set(markers.map((marker) => marker.name))].sort(); + function inputDetails(serialized: ComputeSerialized): { readonly details?: Readonly>; } { if (serialized.input === undefined) return {}; return { details: { - input: serialized.input.value, + input: describeDocument(JSON.parse(serialized.input.value)), ...(serialized.input.absent.length > 0 ? { absent: serialized.input.absent.join('\n') } : {}), }, }; @@ -195,11 +183,6 @@ export function computeDescriptor( return { serviceId: svc.appId, projectId, endpointDomain: svc.appEndpointDomain }; }), - // Two channels of rows: PARAMS (reserved-param literals JSON-encoded; - // dependency provisioning refs passed through, keeping their ordering - // edge) and the INPUT document (one JSON row per service, secret leaves - // as `$secret` pointers, never a value — ADR-0042). The class/branch - // scope is identical for both. serialize: (ctx, provisioned, config) => Effect.gen(function* () { const { address, node, graph } = ctx; @@ -264,24 +247,18 @@ export function computeDescriptor( }, ); if (inputRow !== undefined) { - // Each `$secret` pointer in the document names an OPERATOR-owned - // platform variable Composer never writes, so its rotation shows up - // only as that variable's `updatedAt`. const inputValue = envValue(inputRow.value); rows.push({ record: yield* Prisma.EnvironmentVariable(resourceId(address, 'env', inputRow.key), { project: projectId, key: inputRow.key, - // The defaults-applied document — secret leaves are `$secret` - // pointers, generated leaves are `$generated` pointers, naming - // platform vars, never values (ADR-0042). value: inputValue, class: cls, ...branch, }).pipe(asOwnRow, fromLegacyId(legacyEnvVarId(legacyInputKey(app, address)))), key: inputRow.key, value: inputValue, - pointers: inputRow.secrets, + pointers: sortedMarkerNames(inputRow.markers), }); // Each generated leaf: generate its value ONCE (the resource keeps it // stable across redeploys via its persisted output) and provision it diff --git a/packages/1-prisma-cloud/1-extensions/target/src/env-key-clash.ts b/packages/1-prisma-cloud/1-extensions/target/src/env-key-clash.ts index dffdee560..e16a55354 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/env-key-clash.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/env-key-clash.ts @@ -5,15 +5,11 @@ * Delete this file with `legacy-resource-ids.ts`. */ import type { Graph, ServiceNode } from '@internal/core'; +import { INPUT_KEY_NAME } from '@internal/input-document'; +import { configKey } from './config-key.ts'; import { addressBelowApp } from './legacy-resource-ids.ts'; import { RESERVED_PROVIDER_PARAMS } from './provider-params.ts'; -import { - configKey, - generatedParamVarName, - INPUT_KEY_NAME, - paramEntries, - serializeInput, -} from './serializer.ts'; +import { generatedParamVarName, paramEntries, serializeInput } from './serializer.ts'; export interface EnvKeyClash { /** The variable both rows are written under. */ diff --git a/packages/1-prisma-cloud/1-extensions/target/src/exports/index.ts b/packages/1-prisma-cloud/1-extensions/target/src/exports/index.ts index fb8f9f88a..d767df03e 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/exports/index.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/exports/index.ts @@ -8,6 +8,7 @@ export type { BucketConfig } from '../bucket.ts'; export { bucket, bucketContract } from '../bucket.ts'; export { ComputeService, compute } from '../compute.ts'; +export { configKey } from '../config-key.ts'; export type { GeneratedParamOptions } from '../param.ts'; export { envParam, @@ -35,5 +36,4 @@ export type { CredentialsConfig, CredentialsContract } from '../s3-credentials.t export { credentialsContract, s3Credentials } from '../s3-credentials.ts'; export { s3StoreService } from '../s3-store.ts'; export { envSecret, secretName } from '../secret.ts'; -export { configKey } from '../serializer.ts'; export { STREAMS_API_KEY, STREAMS_API_KEY_ENV, streamsApiKeyNeed } from '../streams-keys.ts'; diff --git a/packages/1-prisma-cloud/1-extensions/target/src/graph-check.ts b/packages/1-prisma-cloud/1-extensions/target/src/graph-check.ts new file mode 100644 index 000000000..18adbfa22 --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/graph-check.ts @@ -0,0 +1,53 @@ +import { type Graph, GraphCheckError, isParamSource, isSecretSource } from '@internal/core'; +import { isEnvParamSource } from './param.ts'; + +interface EnvParamUse { + readonly name: string; + readonly where: string; +} + +function bindingUses(binding: unknown, serviceAddress: string, path: string, out: EnvParamUse[]) { + if (isEnvParamSource(binding)) { + const name = binding.payload.name; + const field = path === '' ? '' : `, field "${path}"`; + out.push({ + name, + where: `envParam('${name}') in the input of service "${serviceAddress}"${field}`, + }); + return; + } + if (isParamSource(binding) || isSecretSource(binding)) return; + if (typeof binding !== 'object' || binding === null) return; + for (const [key, member] of Object.entries(binding)) { + bindingUses(member, serviceAddress, path === '' ? key : `${path}.${key}`, out); + } +} + +function envParamUses(graph: Graph): readonly EnvParamUse[] { + const uses: EnvParamUse[] = []; + for (const { serviceAddress, binding } of graph.inputBindings) { + bindingUses(binding, serviceAddress, '', uses); + } + for (const { serviceAddress, slot, binding } of graph.params) { + if (!isEnvParamSource(binding)) continue; + const name = binding.payload.name; + uses.push({ + name, + where: `envParam('${name}') in param "${slot}" of service "${serviceAddress}"`, + }); + } + return uses; +} + +export function checkGraph(graph: Graph): void { + const rootFields = new Map( + graph.appConfigurationRead.rootInput.map((field) => [field.path[0], field]), + ); + for (const use of envParamUses(graph)) { + const field = rootFields.get(use.name); + if (field === undefined) continue; + throw new GraphCheckError( + `${use.where} and input.${field.path.join('.')} of root module "${graph.root.id}" both use the name "${use.name}". A name has one meaning in an app: read "${use.name}" either with envParam() or as a field of the root's input, not both, or give one of them a different name.`, + ); + } +} diff --git a/packages/1-prisma-cloud/1-extensions/target/src/legacy-resource-ids.ts b/packages/1-prisma-cloud/1-extensions/target/src/legacy-resource-ids.ts index 65e6b2b8a..e1b43efd4 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/legacy-resource-ids.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/legacy-resource-ids.ts @@ -5,7 +5,8 @@ * Delete this file once no such environment remains. */ import { renamedFrom } from 'alchemy/Rename'; -import { configKey, generatedParamVarName, inputKey } from './serializer.ts'; +import { configKey } from './config-key.ts'; +import { generatedParamVarName, inputKey } from './serializer.ts'; /** The address without its first segment, the app name: `shop.auth.api` in the app `shop` is `auth.api`. */ export function addressBelowApp(address: string, app: string): string { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/preflight.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/preflight.ts index e82f6977f..094446dd5 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/preflight.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/preflight.ts @@ -3,7 +3,8 @@ * preflight uses (`preflight-names.ts`), with dev's value-sourcing policy. * Every value comes from what the command read: secrets from the app secrets, env-sourced params from the app configuration. An unset or empty secret is left out of the * service's input; it is an error only when the service's schema rejects - * its absence. An unset or empty env-sourced param is always an error. + * its absence or an array position reads it. An unset or empty per-key env-sourced param takes its default + * or is left out when optional; it is an error only when it is required. * * Control-plane only, runs in the CLI parent (no CliError import — see * container.ts). Reads no Management API — dev is credential-free. @@ -11,7 +12,11 @@ import type { Graph, ServiceNode } from '@internal/core'; import { inputManifest } from '@internal/core'; -import type { PreflightInput, PreflightPayload } from '@internal/core/config'; +import type { + PlatformVariableReader, + PreflightInput, + PreflightPayload, +} from '@internal/core/config'; import { DEV_DIR, devEnvironmentDir } from '@internal/core/config'; import { movePreEnvironmentDevState, @@ -20,23 +25,15 @@ import { } from '@internal/local-target'; import { configurationText } from '../configuration-text.ts'; import { serializeDevPreflightPayload } from '../control/preflight-payload.ts'; -import { - collectPreflightNames, - envParamNotTextError, - inputSecretNames, - type PreflightName, -} from '../preflight-names.ts'; +import { addReader, missingVariablesText } from '../missing-variables-text.ts'; +import { envParamBindings, envParamNotTextError, inputSecretNames } from '../preflight-names.ts'; import { checkPublicUrls, publicUrlStrategies } from '../public-url.ts'; -import { inputValidationFailure } from '../serializer.ts'; - -interface MissingSecrets { - /** One name the schema requires, or several of which it requires some. */ - readonly names: readonly string[]; - readonly serviceAddress: string; - readonly message: string; -} - -type Env = Readonly>; +import { + checkRequirements, + type MissingVariables, + paramIsRequired, + requiredParam, +} from '../required-variables.ts'; const isSet = (value: string | undefined): value is string => value !== undefined && value.length > 0; @@ -51,21 +48,6 @@ function howToSet(environment: string): string { ); } -function missingEnvParamsSection(missing: readonly PreflightName[]): string { - const lines = missing.map((m) => ` - ${m.name} (required by service "${m.address}")`); - return `${missing.length} env-sourced param(s) are not set:\n${lines.join('\n')}`; -} - -function missingSecretsSection(missing: readonly MissingSecrets[]): string { - const lines = missing.map((m) => - m.names.length === 1 - ? ` - ${m.names[0]} (required by service "${m.serviceAddress}": ${m.message})` - : ` - some of ${m.names.join(', ')} (service "${m.serviceAddress}" needs some of them: ` + - `${m.message})`, - ); - return `required secrets are not set:\n${lines.join('\n')}`; -} - function serviceNodeAt(graph: Graph, address: string): ServiceNode { const found = graph.nodes.find((entry) => entry.id === address)?.node; if (found?.kind !== 'service') { @@ -74,30 +56,6 @@ function serviceNodeAt(graph: Graph, address: string): ServiceNode { return found; } -/** The secrets this service's schema cannot do without, among the ones that are unset. */ -function requiredSecrets( - node: ServiceNode, - serviceAddress: string, - binding: unknown, - unset: readonly string[], - env: Env, -): MissingSecrets[] { - const withoutAll = inputValidationFailure(node, serviceAddress, binding, { - env, - absentSecrets: new Set(unset), - }); - if (withoutAll === undefined) return []; - const required: MissingSecrets[] = []; - for (const name of unset) { - const message = inputValidationFailure(node, serviceAddress, binding, { - env, - absentSecrets: new Set([name]), - }); - if (message !== undefined) required.push({ names: [name], serviceAddress, message }); - } - return required.length > 0 ? required : [{ names: unset, serviceAddress, message: withoutAll }]; -} - /** HMAC-SHA256 under a local random key, so the fingerprints in Alchemy state cannot be checked against guessed values without the key file. */ async function fingerprint(key: CryptoKey, value: string): Promise { const digest = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(value)); @@ -125,7 +83,8 @@ export async function runDevPreflight(input: PreflightInput): Promise(); const secretNames = new Set(); const pointed = new Map(); - const missingSecrets: MissingSecrets[] = []; + const required: MissingVariables[] = []; + const readers = new Map(); const invalidInputs: string[] = []; for (const { serviceAddress, binding } of inputManifest(input.graph)) { const names = inputSecretNames(binding, serviceAddress); @@ -138,32 +97,43 @@ export async function runDevPreflight(input: PreflightInput): Promise - envParamNotTextError(param.name, param.address, kind), + envParamNotTextError(param.name, param.serviceAddress, kind), ); - if (isSet(value)) pointed.set(param.name, value); - else missingParams.push(param); + if (isSet(value)) { + pointed.set(param.name, value); + continue; + } + if (!paramIsRequired(serviceNodeAt(input.graph, param.serviceAddress), param.param)) continue; + addReader(readers, param.name, { + kind: 'param', + service: param.serviceAddress, + param: param.param, + }); + required.push(requiredParam(param)); } - const missing = [ - ...(missingParams.length > 0 ? [missingEnvParamsSection(missingParams)] : []), - ...(missingSecrets.length > 0 ? [missingSecretsSection(missingSecrets)] : []), - ]; + const missing = required.length > 0 ? [missingVariablesText(required, readers)] : []; const sections = [ ...invalidInputs, ...missing, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/map-with-concurrency.ts b/packages/1-prisma-cloud/1-extensions/target/src/map-with-concurrency.ts new file mode 100644 index 000000000..e39afa370 --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/map-with-concurrency.ts @@ -0,0 +1,22 @@ +export async function mapWithConcurrency( + items: readonly T[], + limit: number, + task: (item: T) => Promise, +): Promise { + const results: R[] = []; + const queue = items.entries(); + let failed = false; + const worker = async (): Promise => { + for (const [index, item] of queue) { + if (failed) return; + try { + results[index] = await task(item); + } catch (cause) { + failed = true; + throw cause; + } + } + }; + await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker)); + return results; +} diff --git a/packages/1-prisma-cloud/1-extensions/target/src/missing-variables-text.ts b/packages/1-prisma-cloud/1-extensions/target/src/missing-variables-text.ts new file mode 100644 index 000000000..39a742b55 --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/missing-variables-text.ts @@ -0,0 +1,55 @@ +import type { PlatformVariableReader } from '@internal/core/config'; +import { platformVariableReaderText } from '@internal/core/config'; +import type { MissingVariables } from './required-variables.ts'; + +export function addReader( + readers: Map, + name: string, + reader: PlatformVariableReader, +): void { + const list = readers.get(name); + if (list === undefined) readers.set(name, [reader]); + else list.push(reader); +} + +export const missingVariableNames = (required: readonly MissingVariables[]): string[] => [ + ...new Set(required.flatMap((entry) => entry.names)), +]; + +const readersText = (readers: readonly PlatformVariableReader[]): string => + readers.length === 0 ? '' : ` (${readers.map(platformVariableReaderText).join('; ')})`; + +export function missingVariablesText( + required: readonly MissingVariables[], + readers: ReadonlyMap, +): string { + const count = missingVariableNames(required).length; + const reasons = new Map(); + const groups: MissingVariables[] = []; + for (const entry of required) { + const [only] = entry.names; + if (entry.names.length !== 1 || only === undefined) { + groups.push(entry); + continue; + } + const why = reasons.get(only) ?? []; + if (entry.decidedBy === 'rule') { + why.push(` required by service "${entry.serviceAddress}": ${entry.message}`); + } + reasons.set(only, why); + } + const lines = [ + ...[...reasons].map(([name, why]) => + [` - ${name}${readersText(readers.get(name) ?? [])}`, ...why].join('\n'), + ), + ...groups.map( + (entry) => + ` - at least one of ${entry.names.join(', ')} (service "${entry.serviceAddress}" needs at least one of them: ${entry.message})`, + ), + ]; + const countLine = + count === 1 + ? '1 platform variable a service needs has no value:' + : `${count} platform variables a service needs have no value:`; + return `${countLine}\n${lines.join('\n')}`; +} diff --git a/packages/1-prisma-cloud/1-extensions/target/src/param.ts b/packages/1-prisma-cloud/1-extensions/target/src/param.ts index c5bc23357..2f21dc4c5 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/param.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/param.ts @@ -9,6 +9,7 @@ import { } from '@internal/core'; import { blindCast } from '@internal/foundation/casts'; import type { SecretString } from '@internal/foundation/secret'; +import { assertVariableName } from './variable-name.ts'; /** * Brands the payload `envParam` builds. Core's `paramSource()` is a public @@ -27,44 +28,21 @@ export interface EnvParamPayload { readonly name: string; } -const RESERVED_PARAM_PREFIX = 'COMPOSER_'; -/** The names Prisma Cloud owns: it seeds and manages them, so Composer never binds one. */ -const PLATFORM_OWNED_PARAM_NAMES: ReadonlySet = new Set([ - 'DATABASE_URL', - 'DATABASE_URL_POOLED', -]); - /** * Binds a param slot to a named Prisma Cloud platform env var — the non-secret * sibling of `envSecret` (spec: env-sourced config params). The platform * injects the value into the running instance per environment; the param's own - * schema validates it at boot, unredacted. The name may not use the - * framework's reserved `COMPOSER_` prefix or the platform-owned - * `DATABASE_URL(_POOLED)` keys — same parity as `envSecret`. It returns a + * schema validates it at boot, unredacted. The name is an upper-case letter, + * then upper-case letters, digits and underscores. It returns a * value known at deploy: where a module's input or a resolve() function needs * its real value, that is the variable's value in the app configuration: what the * `config` adapters in prisma.config.ts return, by default the process * environment. + * + * @deprecated Only in a module or service `input` binding, where a root input field replaces it. Binding a per-key param with envParam() is not deprecated. */ export function envParam(name: string): ParamSource & Value { - if (typeof name !== 'string' || name.length === 0) { - throw new Error( - "envParam() requires a non-empty platform env-var name, e.g. envParam('APP_ORIGIN').", - ); - } - if (name.startsWith(RESERVED_PARAM_PREFIX)) { - throw new Error( - `envParam name "${name}" may not start with "${RESERVED_PARAM_PREFIX}" — that prefix is ` + - "reserved for the framework's own generated config keys.", - ); - } - if (PLATFORM_OWNED_PARAM_NAMES.has(name)) { - throw new Error( - `envParam name "${name}" is reserved — ${[...PLATFORM_OWNED_PARAM_NAMES].join(' and ')} ` + - 'are seeded and managed by Prisma Cloud itself, so the framework refuses to bind them. ' + - 'Declare the database your service uses and read its url from that connection.', - ); - } + assertVariableName(name, 'envParam'); return paramSource( { [PRISMA_CLOUD_PARAM_SOURCE]: true, name }, { label: `envParam('${name}')`, configurationKey: name }, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/preflight-names.ts b/packages/1-prisma-cloud/1-extensions/target/src/preflight-names.ts index 399d4e054..ffd2ec299 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/preflight-names.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/preflight-names.ts @@ -1,20 +1,5 @@ -/** - * The platform env-var NAMES deploy preflight and dev preflight must both - * check — extracted so the two walks (`runPreflight`, `runDevPreflight`) - * cannot drift (local-dev spec § 5). Deploy checks secrets and env-sourced - * params identically (one platform-existence check per name); dev applies a - * DIFFERENT policy to each (a missing secret is an error only when the - * service's schema rejects its absence, a missing env-sourced param always - * is), so the two lists stay separate here rather than merged into one. - * - * Secrets are the `envSecret` leaves of every service's input binding - * (ADR-0042); env-sourced params are the `envParam`-bound reserved params. - * Secrets also include every secret source the graph names elsewhere - * (secretSourceManifest): one in a module's input that no service receives. - */ - -import type { Graph, SecretSourceUse } from '@internal/core'; -import { inputManifest, isSecretSource, paramManifest, secretSourceManifest } from '@internal/core'; +import type { Graph } from '@internal/core'; +import { isSecretSource, paramManifest } from '@internal/core'; import { isEnvParamSource, isGeneratedParamSource, @@ -23,43 +8,18 @@ import { } from './param.ts'; import { secretName } from './secret.ts'; -export interface PreflightName { - readonly name: string; - /** The node that uses the name: a service whose binding holds it, or, when no service receives it, the module whose input holds it. */ - readonly address: string; - readonly nodeKind: SecretSourceUse['nodeKind']; -} - -/** An env-sourced param's app configuration value that preflight cannot write to a platform variable. */ export const envParamNotTextError = (name: string, address: string, kind: string): Error => new Error( - `The \`config\` adapters return ${kind} for "${name}", not text, so preflight cannot set the platform variable envParam('${name}') of service "${address}" reads. Make the adapter return a string.`, + `The \`config\` adapters return ${kind} for "${name}", not text, so prisma dev cannot give service "${address}" the value envParam('${name}') reads. Make the adapter return a string.`, ); -export interface PreflightNames { - /** Every secret source in a service's input binding or in `graph.secretSources`, deduped by name; the first entry wins the reported address. */ - readonly secrets: readonly PreflightName[]; - /** Every env-sourced param binding's platform var name (envParam-bound only — a literal-bound param never touches the platform), deduped the same way. */ - readonly envParams: readonly PreflightName[]; -} - -function dedupedNames(entries: Iterable): readonly PreflightName[] { - const byName = new Map(); - for (const entry of entries) { - if (!byName.has(entry.name)) byName.set(entry.name, entry); - } - return [...byName.values()]; -} +const composerWritesItsValue = (binding: unknown): boolean => + isGeneratedParamSource(binding) || isOriginParamSource(binding); -/** Every `envSecret` leaf of one input binding: its platform name, found by the same dumb recursive descent the serializer uses (ADR-0042). */ function collectSecretLeafNames(binding: unknown, serviceAddress: string, out: string[]): void { - // A generated leaf provisions its own platform var at deploy — nothing for an - // operator to seed, nothing to check here. Skip it before the object descent - // (it is a ParamSource object, so an unskipped walk would recurse pointlessly - // into its payload). - if (isGeneratedParamSource(binding) || isOriginParamSource(binding)) return; + if (composerWritesItsValue(binding)) return; if (isSecretSource(binding)) { - out.push(secretName(binding, `an input-binding secret leaf of service "${serviceAddress}"`)); + out.push(secretName(binding, `a secret in the input of service "${serviceAddress}"`)); return; } if (typeof binding !== 'object' || binding === null) return; @@ -67,36 +27,24 @@ function collectSecretLeafNames(binding: unknown, serviceAddress: string, out: s for (const member of members) collectSecretLeafNames(member, serviceAddress, out); } -/** The distinct `envSecret` names one service's input binding points at. */ export function inputSecretNames(binding: unknown, serviceAddress: string): readonly string[] { const leafNames: string[] = []; collectSecretLeafNames(binding, serviceAddress, leafNames); return [...new Set(leafNames)]; } -/** Walks each service's input binding for `envSecret` leaves, then `graph.secretSources`, and `graph.params` for env-sourced params (ADR-0042). */ -export function collectPreflightNames(graph: Graph): PreflightNames { - const secretEntries: PreflightName[] = []; - for (const { serviceAddress, binding } of inputManifest(graph)) { - for (const name of inputSecretNames(binding, serviceAddress)) { - secretEntries.push({ name, address: serviceAddress, nodeKind: 'service' }); - } - } - for (const use of secretSourceManifest(graph)) { - secretEntries.push({ - name: secretName(use.source, `a secret source of ${use.nodeKind} "${use.address}"`), - address: use.address, - nodeKind: use.nodeKind, - }); - } - const envParams = dedupedNames( - paramManifest(graph) - .filter((binding) => isEnvParamSource(binding.binding)) - .map((binding) => ({ - name: paramName(binding), - address: binding.serviceAddress, - nodeKind: 'service' as const, - })), - ); - return { secrets: dedupedNames(secretEntries), envParams }; +export interface EnvParamBinding { + readonly name: string; + readonly serviceAddress: string; + readonly param: string; +} + +export function envParamBindings(graph: Graph): readonly EnvParamBinding[] { + return paramManifest(graph) + .filter((binding) => isEnvParamSource(binding.binding)) + .map((binding) => ({ + name: paramName(binding), + serviceAddress: binding.serviceAddress, + param: binding.slot, + })); } diff --git a/packages/1-prisma-cloud/1-extensions/target/src/preflight.ts b/packages/1-prisma-cloud/1-extensions/target/src/preflight.ts index 1168145ea..0cccf8d51 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/preflight.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/preflight.ts @@ -1,28 +1,10 @@ -/** - * Deploy preflight (ADR-0029, extended for env-sourced params): before Alchemy - * runs, verify every secret binding AND every env-sourced param binding in the - * app's manifests (each a platform env-var NAME) exists on Prisma Cloud for - * the target environment. A literal-bound param never touches the platform — only - * `envParam(...)` sources are checked here. - * A name absent on the platform but present in what the command read (the app secrets for a secret, the app configuration for an env-sourced param) is provisioned - * via a direct Management API POST — NEVER an Alchemy resource, so the value - * never lands in hosted deploy state. A name absent from both fails the deploy, - * listing exactly what is missing and where to set it. - * - * Control-plane only (imported by control.ts → prisma.config.ts); runs - * in the CLI parent, on the caller's Management API client when it passed one, - * and otherwise on a client built from env — the same credential path - * `container.ts`'s `ensure`/`locate` use. - * - * It also returns WHEN each of those names was last written. This is the only - * place in a deploy that reads those rows off the platform, so it is where the - * reading belongs; the compute deploy hook folds the timestamps into its - * environment fingerprint so that rotating a secret or an env-sourced param - * out of band ships a new deployment. A timestamp, never a value: env-var - * values are write-only and the API never returns one. - */ -import type { Graph, SecretSourceUse } from '@internal/core'; -import type { PreflightInput } from '@internal/core/config'; +import type { Graph, ServiceNode } from '@internal/core'; +import { inputManifest, secretSourceManifest } from '@internal/core'; +import type { + PlatformVariableReader, + PreflightInput, + UnsetPlatformVariable, +} from '@internal/core/config'; import { blindCast } from '@internal/foundation/casts'; import { drivePagesAsync, @@ -33,13 +15,22 @@ import { } from '@internal/lowering'; import * as Effect from 'effect/Effect'; import * as Layer from 'effect/Layer'; -import { configurationText } from './configuration-text.ts'; import { prismaCloudContainerOf } from './container.ts'; +import type { DeployPreflightPayload } from './control/preflight-payload.ts'; import { type EnvKeyClash, findEnvKeyClashes } from './env-key-clash.ts'; +import { mapWithConcurrency } from './map-with-concurrency.ts'; +import { addReader, missingVariableNames, missingVariablesText } from './missing-variables-text.ts'; import { resolveOrmConfig } from './orm-config.ts'; import { isPostgresResourceNode, requiredPackHeadOf } from './orm-postgres.ts'; -import { collectPreflightNames, envParamNotTextError } from './preflight-names.ts'; +import { envParamBindings, inputSecretNames } from './preflight-names.ts'; import { checkPublicUrls } from './public-url.ts'; +import { + checkRequirements, + type MissingVariables, + paramIsRequired, + requiredParam, +} from './required-variables.ts'; +import { secretName } from './secret.ts'; type EnvClass = 'production' | 'preview'; @@ -52,6 +43,7 @@ interface EnvVarListPage { readonly data: readonly { readonly branchId: string | null; readonly updatedAt: string; + readonly isManagedBySystem?: boolean; }[]; readonly pagination: { readonly nextCursor: string | null; readonly hasMore: boolean }; } @@ -84,45 +76,23 @@ async function listEnvVars( >(res); } -/** What the platform holds for one name: whether it is there at all, and when it last changed. */ -interface PlatformVariable { - readonly exists: boolean; - /** The latest `updatedAt` across every row visible to this environment — undefined when the name is absent. */ +interface ResolvedRow { readonly updatedAt?: string; + readonly managedBySystem: boolean; } -/** - * What the platform holds for `key` in the target environment's scope. `production` - * → any production-class template. Any other environment → a preview template (branchId - * null) OR this branch's own override — the platform's preview materialization - * (pdp-data-model.md). Metadata read only; env-var values are write-only. - * - * The whole list is walked rather than short-circuiting on the first visible - * row, because the newest `updatedAt` across every visible row is the rotation - * signal the compute deploy hook fingerprints on: stopping early would make - * that timestamp depend on where the page boundary happened to fall, and a - * fingerprint that moves for that reason would redeploy for no reason. A key - * with more rows than one page (a template plus many per-branch overrides) is - * rare, so this costs one request in practice. - */ +const LIST_CONCURRENCY = 8; + async function readPlatformVariable( client: ManagementApiClient, projectId: string, branchId: string | undefined, key: string, -): Promise { +): Promise { const cls = classFor(branchId); - const visible = (row: { branchId: string | null }): boolean => - branchId === undefined || row.branchId === null || row.branchId === branchId; - - // The list is paginated: a key with more preview rows (template + many - // per-branch overrides) than one page must be followed to the end, or a - // present name is falsely reported missing. Walked to the end (no - // short-circuit) because the LATEST updatedAt across visible rows feeds - // the deploy fingerprint; bounded (drivePagesAsync) so broken pagination - // fails loudly instead of looping. - let exists = false; - let latest: string | undefined; + let classRow: string | undefined; + let branchRow: string | undefined; + let managedBySystem = false; await drivePagesAsync( `environment variables named "${key}"`, async (cursor) => { @@ -137,18 +107,17 @@ async function readPlatformVariable( }, (data) => { for (const row of data) { - if (!visible(row)) continue; - exists = true; - // Parsed, not string-compared: lexicographic order breaks the moment - // two rows serialize with different precision or UTC designators. - if (latest === undefined || Date.parse(row.updatedAt) > Date.parse(latest)) { - latest = row.updatedAt; - } + if (row.branchId === null) classRow = row.updatedAt; + else if (branchId !== undefined && row.branchId === branchId) branchRow = row.updatedAt; + else continue; + managedBySystem ||= row.isManagedBySystem === true; } return false; }, ); - return latest === undefined ? { exists } : { exists, updatedAt: latest }; + const updatedAt = branchRow ?? classRow; + if (updatedAt === undefined) return undefined; + return { updatedAt, managedBySystem }; } /** Whether `key` has a row in the scope Composer writes its own rows to: production, or the environment's branch. */ @@ -217,47 +186,6 @@ async function pendingEnvKeyClashes( return pending; } -/** - * Provision `key`=`value` directly via the Management API for the target - * environment's scope (a production template for `production`; a preview branch - * override for any other environment — the same scope the pack's config rows are - * written to, through alchemy's `Prisma.EnvironmentVariable`). A 409 means a - * concurrent deploy already provisioned - * it — tolerated. The value is never logged. - */ -async function fillMissing( - client: ManagementApiClient, - projectId: string, - branchId: string | undefined, - key: string, - value: string, - list: ValueList, -): Promise { - const res = await client.POST('/v1/environment-variables', { - body: { - projectId, - class: classFor(branchId), - key, - value, - ...(branchId !== undefined ? { branchId } : {}), - }, - }); - if (res.error !== undefined && res.response.status !== 409) { - throw fillFailedError(key, list, res.error); - } - // The created row's timestamp, so this deploy fingerprints on the same value - // the next one will read back. A 409 (a concurrent deploy won the race) - // returns no row: the name reads as unknown for this run and the next deploy - // picks its timestamp up, which redeploys once — the safe direction. - return res.data?.data.updatedAt; -} - -interface MissingBinding { - readonly name: string; - readonly address: string; - readonly nodeKind: SecretSourceUse['nodeKind']; -} - /** The preflight input typed against this extension's own client — the concrete form of the framework's erased `PreflightInput`. */ export type PrismaCloudPreflightInput = PreflightInput; @@ -271,50 +199,43 @@ const listFailedError = (key: string, error: unknown): Error => `deploy preflight: Prisma Management API error listing "${key}": ${JSON.stringify(error)}.`, ); -type ValueList = 'the app secrets' | 'the app configuration'; - -const fillFailedError = (key: string, list: ValueList, error: unknown): Error => - new Error( - `deploy preflight: failed to set "${key}" on the platform from ${list}: ${JSON.stringify(error)}.`, - ); - /** `value` as one POSIX shell word: single-quoted, each embedded `'` written as `'\''`. */ const shellQuote = (value: string): string => `'${value.replaceAll("'", "'\\''")}'`; -function missingError( - missing: readonly MissingBinding[], +function setCommands( + name: string, projectId: string, branchId: string | undefined, environment: string, +): readonly string[] { + const add = `prisma project env add ${shellQuote(`${name}=`)} --project ${shellQuote(projectId)}`; + return branchId === undefined + ? [`${add} --role production`] + : [`${add} --role preview`, `${add} --role preview --branch ${shellQuote(environment)}`]; +} + +function requiredError( + required: readonly MissingVariables[], + readers: ReadonlyMap, + commandsFor: (name: string) => readonly string[], ): Error { - const lines = missing.map((m) => ` - ${m.name} (used by ${m.nodeKind} "${m.address}")`); - const countPhrase = - missing.length === 1 ? '1 required setting has' : `${missing.length} required settings have`; - const scopeFlag = - branchId === undefined ? '--role production' : `--branch ${shellQuote(environment)}`; - // `project env add` takes a single KEY=VALUE per call. Quoting the pair keeps the - // `` placeholder from being read as a shell redirection. - const commands = missing.map( - (m) => - `prisma project env add ${shellQuote(`${m.name}=`)} --project ${shellQuote(projectId)} ${scopeFlag}`, - ); - const runStep = - commands.length === 1 - ? `Run: ${commands[0]}` - : `Run, once per setting:\n${commands.map((cmd) => ` ${cmd}`).join('\n')}`; - const consoleStep = - branchId === undefined - ? 'add each one under Production. Those values apply when the default branch deploys to production.' - : `add each one under Preview. Preview values apply to every branch deploy, including "${environment}".`; + const names = missingVariableNames(required); + const commands = names.flatMap(commandsFor).map((command) => ` - ${command}`); return new Error( - `Deploy failed. ${countPhrase} no value:\n` + - `${lines.join('\n')}\n\n` + - 'Set the value in one of these two places, then deploy again:\n' + - ` - ${runStep}\n` + - ` - Or in the Prisma Console: open the project, go to Environment variables, and ${consoleStep}`, + `Deploy failed. ${missingVariablesText(required, readers)}\n\n` + + `Set ${names.length === 1 ? 'it' : 'them'} on the platform, then deploy again:\n` + + commands.join('\n'), ); } +const managedBySystemError = (names: readonly string[]): Error => + new Error( + `Deploy failed. ${names.map((name) => `"${name}"`).join(', ')} ${names.length === 1 ? 'is a platform variable' : 'are platform variables'} that Prisma Cloud manages itself, so the app cannot read ${names.length === 1 ? 'it' : 'them'} as its own. Rename ${names.length === 1 ? 'it' : 'them'} in the app.`, + ); + +const invalidInputError = (serviceAddress: string, message: string): Error => + new Error(`invalid input for service "${serviceAddress}": ${message}`); + function envKeyClashError( clashes: readonly EnvKeyClash[], projectId: string, @@ -354,93 +275,123 @@ async function managementClient(): Promise { ); } -/** - * The Prisma Cloud extension's `preflight`. Every secret source the graph - * names is checked, including one in a module's input that no service - * receives. Uses the shared name collector (`collectPreflightNames`, - * ADR-0042): every secret - * source in a service's input binding or in `graph.secretSources`, deduped by - * name with the first entry winning the reported address, plus - * `paramManifest` filtered to env-sourced reserved params. It checks each - * platform name against the platform, fills from the - * app configuration and the app secrets where possible, and fails loudly on anything absent from both. - * `envParam` leaves of an input binding are NOT checked here — they resolve - * from the app configuration at serialize, and an unset one is an omitted key the - * schema arbitrates. Uses the caller's client (`input.credentials`), then an - * injected one for tests, then a client built from env. - */ +function serviceNodeAt(graph: Graph, address: string): ServiceNode { + const found = graph.nodes.find((entry) => entry.id === address)?.node; + if (found?.kind !== 'service') { + throw new Error(`deploy preflight: no service node at address "${address}".`); + } + return found; +} + export async function runPreflight( input: PrismaCloudPreflightInput, deps?: { readonly client?: ManagementApiClient }, -): Promise> { +): Promise { checkPublicUrls(input.graph); const { projectId, branchId } = prismaCloudContainerOf(input.container); - // One check per platform NAME (many leaves/services, secret or param, may - // bind the same one). Every bound secret leaf is required — absence is - // expressed by omitting the key from the binding, never by an unprovisioned - // var; a param binding is only checked when it is env-sourced. Deploy treats - // the two lists identically (unlike dev preflight), so they're merged here, - // secrets first. - const collected = collectPreflightNames(input.graph); - const names = new Map(); - const valueFor = new Map string | undefined }>(); - for (const meta of collected.secrets) { - if (names.has(meta.name)) continue; - names.set(meta.name, meta); - valueFor.set(meta.name, { list: 'the app secrets', read: () => input.secrets[meta.name] }); - } - for (const meta of collected.envParams) { - if (names.has(meta.name)) continue; - names.set(meta.name, meta); - valueFor.set(meta.name, { - list: 'the app configuration', - read: () => - configurationText(input.configuration, meta.name, (kind) => - envParamNotTextError(meta.name, meta.address, kind), - ), - }); - } + const services = inputManifest(input.graph).map(({ serviceAddress, binding }) => ({ + serviceAddress, + binding, + names: inputSecretNames(binding, serviceAddress), + })); + const params = envParamBindings(input.graph); + const unread = secretSourceManifest(input.graph).flatMap((use) => + use.nodeKind === 'service' + ? [] + : [ + { + name: secretName(use.source, `a secret source of ${use.nodeKind} "${use.address}"`), + reader: { kind: 'unread', node: use.address, nodeKind: use.nodeKind } as const, + }, + ], + ); + const names = [ + ...new Set([ + ...services.flatMap((service) => service.names), + ...unread.map((use) => use.name), + ...params.map((param) => param.name), + ]), + ]; const clashes = findEnvKeyClashes(input.graph); - if (names.size === 0 && clashes.length === 0) { - input.report?.({ checked: 0, filled: 0, missing: 0 }); - return new Map(); + if (names.length === 0 && clashes.length === 0) { + input.report?.({ checked: 0, unset: [] }); + return { updatedAt: new Map() }; } const client = input.credentials?.client ?? deps?.client ?? (await managementClient()); const pending = await pendingEnvKeyClashes(client, projectId, branchId, clashes); if (pending.length > 0) throw envKeyClashError(pending, projectId, branchId, input.environment); - const missing: MissingBinding[] = []; + + const rows = await mapWithConcurrency(names, LIST_CONCURRENCY, (name) => + readPlatformVariable(client, projectId, branchId, name), + ); const updatedAt = new Map(); - let filled = 0; - for (const meta of names.values()) { - const platform = await readPlatformVariable(client, projectId, branchId, meta.name); - if (platform.exists) { - if (platform.updatedAt !== undefined) updatedAt.set(meta.name, platform.updatedAt); - continue; + const unset = new Set(); + const managedBySystem: string[] = []; + names.forEach((name, index) => { + const row = rows[index]; + if (row?.updatedAt === undefined) { + unset.add(name); + return; } - const source = valueFor.get(meta.name); - const value = source?.read(); - if (source !== undefined && value !== undefined && value.length > 0) { - const filledAt = await fillMissing( - client, - projectId, - branchId, - meta.name, - value, - source.list, - ); - if (filledAt !== undefined) { - updatedAt.set(meta.name, filledAt); - filled += 1; - } - continue; + updatedAt.set(name, row.updatedAt); + if (row.managedBySystem) managedBySystem.push(name); + }); + if (managedBySystem.length > 0) throw managedBySystemError(managedBySystem); + if (unset.size === 0) { + input.report?.({ checked: names.length, unset: [] }); + return { updatedAt }; + } + + const readers = new Map(); + const required: MissingVariables[] = []; + for (const { serviceAddress, binding, names: serviceNames } of services) { + const unsetHere = serviceNames.filter((name) => unset.has(name)); + if (unsetHere.length === 0) continue; + const node = serviceNodeAt(input.graph, serviceAddress); + const decided = checkRequirements( + node, + serviceAddress, + binding, + unsetHere, + input.configuration, + ); + if (decided.kind === 'invalid') throw invalidInputError(serviceAddress, decided.message); + for (const marker of decided.markers) { + if (!unset.has(marker.name)) continue; + addReader(readers, marker.name, { + kind: 'field', + service: serviceAddress, + path: marker.path, + }); } - missing.push(meta); + required.push(...decided.missing); } - input.report?.({ checked: names.size, filled, missing: missing.length }); - if (missing.length > 0) throw missingError(missing, projectId, branchId, input.environment); - return updatedAt; + for (const use of unread) { + if (unset.has(use.name)) addReader(readers, use.name, use.reader); + } + for (const param of params) { + if (!unset.has(param.name)) continue; + addReader(readers, param.name, { + kind: 'param', + service: param.serviceAddress, + param: param.param, + }); + if (paramIsRequired(serviceNodeAt(input.graph, param.serviceAddress), param.param)) { + required.push(requiredParam(param)); + } + } + + const commandsFor = (name: string) => setCommands(name, projectId, branchId, input.environment); + if (required.length > 0) throw requiredError(required, readers, commandsFor); + const findings: UnsetPlatformVariable[] = [...unset].sort().map((name) => ({ + name, + readers: readers.get(name) ?? [], + commands: commandsFor(name), + })); + input.report?.({ checked: names.length, unset: findings }); + return { updatedAt }; } /** diff --git a/packages/1-prisma-cloud/1-extensions/target/src/required-variables.ts b/packages/1-prisma-cloud/1-extensions/target/src/required-variables.ts new file mode 100644 index 000000000..322f4f01f --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/required-variables.ts @@ -0,0 +1,90 @@ +import type { ServiceNode } from '@internal/core'; +import type { PlatformMarkerUse } from '@internal/input-document'; +import { inputValidationFailure, serializeInput } from './serializer.ts'; + +export interface MissingVariables { + readonly names: readonly string[]; + readonly serviceAddress: string; + readonly message: string; + /** `schema` when the service's schema failed without them; `rule` when a rule outside the schema requires them. */ + readonly decidedBy: 'schema' | 'rule'; +} + +export type RequirementCheck = + | { readonly kind: 'invalid'; readonly message: string } + | { + readonly kind: 'checked'; + readonly missing: readonly MissingVariables[]; + readonly markers: readonly PlatformMarkerUse[]; + }; + +const ARRAY_POSITION_MESSAGE = + 'an array position reads it, and an array position cannot be left out'; + +export function checkRequirements( + node: ServiceNode, + serviceAddress: string, + binding: unknown, + unset: readonly string[], + env: Readonly>, +): RequirementCheck { + const invalid = inputValidationFailure(node, serviceAddress, binding, { + env, + absentSecrets: new Set(), + }); + if (invalid !== undefined) return { kind: 'invalid', message: invalid }; + const markers = serializeInput(node, serviceAddress, binding, { env })?.markers ?? []; + const unsetNames = new Set(unset); + const inArray = new Set( + markers.filter((marker) => marker.inArray && unsetNames.has(marker.name)).map((m) => m.name), + ); + const missing: MissingVariables[] = [...inArray].map((name) => ({ + names: [name], + serviceAddress, + message: ARRAY_POSITION_MESSAGE, + decidedBy: 'rule', + })); + const rest = unset.filter((name) => !inArray.has(name)); + const checked = (found: readonly MissingVariables[]): RequirementCheck => ({ + kind: 'checked', + missing: [...missing, ...found], + markers, + }); + if (rest.length === 0) return checked([]); + const withoutAll = inputValidationFailure(node, serviceAddress, binding, { + env, + absentSecrets: new Set(rest), + }); + if (withoutAll === undefined) return checked([]); + const alone: MissingVariables[] = []; + for (const name of rest) { + const message = inputValidationFailure(node, serviceAddress, binding, { + env, + absentSecrets: new Set([name]), + }); + if (message !== undefined) { + alone.push({ names: [name], serviceAddress, message, decidedBy: 'schema' }); + } + } + return checked( + alone.length > 0 + ? alone + : [{ names: rest, serviceAddress, message: withoutAll, decidedBy: 'schema' }], + ); +} + +export function paramIsRequired(node: ServiceNode, param: string): boolean { + const declared = node.params[param]; + return declared !== undefined && declared.default === undefined && declared.optional !== true; +} + +export const requiredParam = (param: { + readonly name: string; + readonly serviceAddress: string; + readonly param: string; +}): MissingVariables => ({ + names: [param.name], + serviceAddress: param.serviceAddress, + message: `per-key param "${param.param}" has no default and is not optional`, + decidedBy: 'rule', +}); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/secret.ts b/packages/1-prisma-cloud/1-extensions/target/src/secret.ts index 406850c24..da5b7b6c5 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/secret.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/secret.ts @@ -1,6 +1,7 @@ import { type Pointer, type RuntimeValue, type SecretSource, secretSource } from '@internal/core'; import { blindCast } from '@internal/foundation/casts'; import type { SecretString } from '@internal/foundation/secret'; +import { assertVariableName } from './variable-name.ts'; /** * Brands the payload `envSecret` builds. Core's `secretSource()` is a public @@ -19,42 +20,18 @@ export interface EnvSecretPayload { readonly name: string; } -const RESERVED_SECRET_PREFIX = 'COMPOSER_'; -/** The names Prisma Cloud owns: it seeds and manages them, so Composer never binds one. */ -const PLATFORM_OWNED_SECRET_NAMES: ReadonlySet = new Set([ - 'DATABASE_URL', - 'DATABASE_URL_POOLED', -]); - /** * A value pointing at a named Prisma Cloud platform secret variable (ADR-0029), * dereferenced at boot. Only the name is carried; the value is provisioned - * out-of-band. The name may not use the framework's reserved `COMPOSER_` prefix - * or the platform-owned `DATABASE_URL(_POOLED)` keys. It returns a value known + * out-of-band. The name is an upper-case letter, then upper-case letters, + * digits and underscores. It returns a value known * only at boot, which a resolve() function cannot list, and which throws when * turned into text. */ export function envSecret( name: string, ): SecretSource & RuntimeValue { - if (typeof name !== 'string' || name.length === 0) { - throw new Error( - "envSecret() requires a non-empty platform env-var name, e.g. envSecret('STRIPE_SECRET_KEY').", - ); - } - if (name.startsWith(RESERVED_SECRET_PREFIX)) { - throw new Error( - `envSecret name "${name}" may not start with "${RESERVED_SECRET_PREFIX}" — that prefix is ` + - "reserved for the framework's own generated config keys.", - ); - } - if (PLATFORM_OWNED_SECRET_NAMES.has(name)) { - throw new Error( - `envSecret name "${name}" is reserved — ${[...PLATFORM_OWNED_SECRET_NAMES].join(' and ')} ` + - 'are seeded and managed by Prisma Cloud itself, so the framework refuses to bind them. ' + - 'Declare the database your service uses and read its url from that connection.', - ); - } + assertVariableName(name, 'envSecret'); return secretSource>( { [PRISMA_CLOUD_SECRET_SOURCE]: true, name }, { label: `envSecret('${name}')`, secretsKey: name }, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/serializer.ts b/packages/1-prisma-cloud/1-extensions/target/src/serializer.ts index 8fdb77f84..f3e346eda 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/serializer.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/serializer.ts @@ -26,16 +26,27 @@ import type { Config, ConfigParam, Params, ServiceNode } from '@internal/core'; import { isParamSource, isSecretPointer, isSecretSource } from '@internal/core'; import { blindCast } from '@internal/foundation/casts'; import { SecretBox } from '@internal/foundation/secret'; +import { + addressKeySegments, + decodeParamPointer, + escapeKey, + generatedMarker, + INPUT_KEY_NAME, + isParamPointerRow, + type PlatformMarkerUse, + platformMarker, + readDocument, + requiredParamUnsetMessage, + withUnsetVariables, +} from '@internal/input-document'; import type { StandardSchemaV1 } from '@standard-schema/spec'; import { type } from 'arktype'; -import { addressKeySegments, configKey, ORIGIN_KEY_NAME } from './config-key.ts'; +import { configKey, ORIGIN_KEY_NAME } from './config-key.ts'; import { configurationText } from './configuration-text.ts'; import { isEnvParamSource, isGeneratedParamSource, isOriginParamSource } from './param.ts'; import { RESERVED_PROVIDER_PARAMS } from './provider-params.ts'; import { secretName } from './secret.ts'; -export { addressKeySegments, configKey, ORIGIN_KEY_NAME }; - // The ambient environment of whatever runtime hosts the bundle. Declared // structurally so this entry imports no runtime's types. Writable: run() // applies stashRows() here under address-free keys. @@ -98,31 +109,7 @@ function decode(owner: ParamEntry['owner'], raw: string): unknown { return owner === 'service' ? JSON.parse(raw) : raw; } -// ——— Env-sourced param pointer rows: the non-secret sibling of the secret -// pointer channel below. A service's own param is EITHER literal-bound -// (JSON-encoded via `encode` above, unchanged) OR env-source-bound (a pointer -// to a platform var, this marker) for a given deploy — never both — so the -// two need to be told apart from the stored row alone, with no separate -// channel to consult. `JSON.stringify` output always starts with one of -// `"{[-0123456789tfn` (string/object/array/number/bool/null); the marker -// starts with `@`, which no JSON.stringify output can start with, so the two -// never collide. -const PARAM_POINTER_PREFIX = '@composer-param-pointer:'; - -/** True iff `raw` is a param pointer row (as opposed to a JSON-encoded literal). */ -export const isParamPointerRow = (raw: string): boolean => raw.startsWith(PARAM_POINTER_PREFIX); - -/** Builds a param pointer row's stored value from the platform var NAME it points to. */ -export const encodeParamPointer = (name: string): string => `${PARAM_POINTER_PREFIX}${name}`; - -/** Reverses `encodeParamPointer`: the platform var NAME a pointer row points to. */ -export const decodeParamPointer = (raw: string): string => raw.slice(PARAM_POINTER_PREFIX.length); - function coerce(raw: string | undefined, d: ParamEntry, key: string, env: Env): unknown { - // "" is UNRESOLVED, not a value — falls to the default or, if required, is - // a loud boot failure; a NON-EMPTY value that fails its param's declared - // schema is an error regardless of any default (a default substitutes for - // absence, never for garbage). const present = raw !== undefined && raw !== ''; if (!present) { if (d.param.default !== undefined) return d.param.default; @@ -130,7 +117,7 @@ function coerce(raw: string | undefined, d: ParamEntry, key: string, env: Env): throw new Error(`missing required config param "${d.name}" (env ${key})`); } if (d.owner === 'service' && isParamPointerRow(raw)) { - return coerceEnvSourcedParam(raw, d, key, env); + return coerceEnvSourcedParam(raw, d, env); } try { return standardValidateSync(d.param.schema, decode(d.owner, raw)); @@ -140,30 +127,20 @@ function coerce(raw: string | undefined, d: ParamEntry, key: string, env: Env): } } -/** - * Boot resolution for an env-sourced param: double-lookup (pointer → platform - * var), then the param's own schema on the raw string — no JSON decode, and - * no redaction (it's config, not a secret). An UNSET platform var is a loud - * boot failure naming both the param and the platform var; an EMPTY string is - * not special-cased here — it reaches the schema like any other value, so it - * passes iff the schema accepts it (deliberately unlike a literal param's own - * ""-means-absent rule, and unlike a secret's non-empty requirement). - */ -function coerceEnvSourcedParam(raw: string, d: ParamEntry, key: string, env: Env): unknown { +function coerceEnvSourcedParam(raw: string, d: ParamEntry, env: Env): unknown { const platformVar = decodeParamPointer(raw); const value = env[platformVar]; - if (value === undefined) { - throw new Error( - `env-sourced config param "${d.name}" (env ${key} → ${platformVar}) is unset: the platform ` + - `variable "${platformVar}" was not injected — the deploy did not provision it.`, - ); + if (value === undefined || value === '') { + if (d.param.default !== undefined) return d.param.default; + if (d.param.optional === true) return undefined; + throw new Error(requiredParamUnsetMessage(d.name, platformVar)); } try { return standardValidateSync(d.param.schema, value); } catch (cause) { const message = cause instanceof Error ? cause.message : String(cause); throw new Error( - `invalid value for env-sourced config param "${d.name}" (env ${key} → ${platformVar}): ${message}`, + `The param "${d.name}" reads the platform variable "${platformVar}", whose value is invalid: ${message}`, ); } } @@ -180,16 +157,16 @@ export const deserialize = (node: ServiceNode, address: string, env: Env = proce for (const d of paramEntries(node)) { const key = configKey(address, d); const value = coerce(env[key], d, key, env); - if (d.owner === 'service') { - service[d.name] = value; - } else { + let target = service; + if (d.owner !== 'service') { let bucket = inputs[d.owner.input]; if (bucket === undefined) { bucket = {}; inputs[d.owner.input] = bucket; } - bucket[d.name] = value; + target = bucket; } + if (value !== undefined) target[d.name] = value; } return { service, inputs }; @@ -261,21 +238,6 @@ export function stashRows(node: ServiceNode, address: string, env: Env): Record< return rows; } -// ——— Input channel (ADR-0042): ONE self-describing JSON document per service. -// -// Deploy resolves the provision-time input binding (literals as-is, envParam -// leaves read from the app configuration, envSecret leaves as `$secret` pointers), -// validates the resolved object with the service's own Standard Schema -// (each value known only at boot as its pointer — never a value), and writes the -// defaults-applied result to COMPOSER__INPUT. Boot reads that one row, -// swaps each pointer for a redacting box over the named platform var, -// validates again, and hands the app the typed object. Writer and reader -// share the walk below, so they cannot drift. Secret VALUES never enter the -// document. - -/** The input document's own row name; its full key is COMPOSER__INPUT. */ -export const INPUT_KEY_NAME = 'INPUT'; - /** The input document row's key for a service address. */ export const inputKey = (address: string): string => configKey(address, { owner: 'service', name: INPUT_KEY_NAME }); @@ -294,45 +256,6 @@ export const generatedParamVarName = (address: string, path: string): string => return ['COMPOSER', ...segments, ...pathSegments, 'GENERATED'].join('_').toUpperCase(); }; -// The document's two reserved keys, "$secret" and "$generated". User data may -// legitimately contain such a key, so the writer escapes any key matching -// /^\$+(secret|generated)$/ by prefixing one more "$" ("$secret" → "$$secret", -// "$$secret" → "$$$secret"), and the reader strips one "$" back off — a -// round-trip under which only the framework can put a literal single-"$" marker -// in the document. -const SECRET_MARKER = '$secret'; -const GENERATED_MARKER = '$generated'; -const ESCAPABLE_KEY = /^\$+(?:secret|generated)$/; -const ESCAPED_KEY = /^\$\$+(?:secret|generated)$/; - -/** True for a secret pointer as the deploy writes it into the input document: `{ "$secret": "" }` and nothing else. */ -function isWrittenSecretPointer(value: unknown): value is { readonly $secret: string } { - return ( - typeof value === 'object' && - value !== null && - !Array.isArray(value) && - Object.keys(value).length === 1 && - SECRET_MARKER in value && - typeof value[SECRET_MARKER] === 'string' - ); -} - -/** True for the framework's generated pointer: `{ "$generated": "", "redacted": }` and nothing else. The `redacted` facet rides the pointer so boot — which is schema-blind — knows whether to box (§ 4.1). */ -function isGeneratedPointer( - value: unknown, -): value is { readonly $generated: string; readonly redacted: boolean } { - return ( - typeof value === 'object' && - value !== null && - !Array.isArray(value) && - Object.keys(value).length === 2 && - GENERATED_MARKER in value && - typeof value[GENERATED_MARKER] === 'string' && - 'redacted' in value && - typeof value['redacted'] === 'boolean' - ); -} - /** A plain data object (not an array, not a class instance) — the only object shape a binding/document may nest. */ function isPlainObject(value: unknown): value is Record { if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; @@ -352,7 +275,6 @@ export interface GeneratedLeaf { readonly origin?: string; } -/** One pointer var name and the input path that produced it — the collision guard's unit. */ export interface PointerName { readonly path: string; readonly varName: string; @@ -369,6 +291,23 @@ export interface ResolvedInputBinding { const NO_ABSENT_SECRETS: ReadonlySet = new Set(); +function absentArrayPositionError(member: unknown, position: string, address: string): Error { + const key = `input binding key "${position}" of service "${address}"`; + if (isSecretSource(member)) { + return new Error( + `${key} reads the platform variable "${secretName(member, 'an array position')}", which has no value. An array position cannot be left out: set the variable, or bind a literal.`, + ); + } + if (isEnvParamSource(member)) { + return new Error( + `${key} reads "${member.payload.name}" from the app configuration, which has no value for it. An array position cannot be left out: set it where a \`config\` adapter reads it, or bind a literal.`, + ); + } + return new Error( + `${key} resolved to no value, but only envSecret() and envParam() can. This is a framework bug.`, + ); +} + /** How `resolveInputBinding`, `serializeInput` and `inputValidationFailure` read the environment and which secrets they leave out. */ export interface SerializeInputOptions { /** Where `envParam` leaves are read: the app configuration, or in the second process the configuration key text from the app configuration file. The process environment when not given. */ @@ -384,15 +323,6 @@ function withDefaults(options: SerializeInputOptions): Required { const resolvedMember = walk(member, path === '' ? String(index) : `${path}.${index}`); if (resolvedMember === ABSENT) { - throw new Error( - `input binding key "${path}[${index}]" is an env-bound array element whose variable is ` + - 'unset — an array position cannot be omitted; bind a literal or provision the variable.', - ); + throw absentArrayPositionError(member, `${path}[${index}]`, address); } return resolvedMember; }); @@ -536,19 +462,14 @@ export function resolveInputBinding( /** One serialized input document, ready to write as an env row. */ export interface InputDocumentRow { readonly key: string; - /** The JSON document — defaults-applied, secret leaves as `$secret` pointers, generated leaves as `$generated` pointers, secret-free by construction. */ + /** The JSON document: defaults applied, each secret leaf a `$platform` marker, each generated leaf a `$generated` marker, never a secret's value. */ readonly value: string; /** Dot-joined binding paths whose env-bound leaf resolved absent — deploy-report fodder. */ readonly absent: readonly string[]; /** Generated leaves the descriptor must provision (a `GeneratedParam` resource + env row each). */ readonly generated: readonly GeneratedLeaf[]; - /** - * The platform variable each `$secret` pointer in the document names — - * operator-provisioned, never written by Composer. The deploy hook folds - * each one's `updatedAt` into the environment fingerprint, so rotating a - * secret out of band ships a new deployment. - */ - readonly secrets: readonly string[]; + /** Every `$platform` marker the document holds, in document order. Composer does not write the variables they name. */ + readonly markers: readonly PlatformMarkerUse[]; } /** The text a stand-in box holds: known, and not a secret. */ @@ -557,7 +478,7 @@ const STAND_IN_TEXT = 'composer-stand-in-not-a-secret'; /** * Deploy: resolve the binding, judge it with the service's own schema (each * value known only at boot as its pointer), and serialize the - * defaults-applied VALIDATED output, each pointer written as a `$secret` or + * defaults-applied VALIDATED output, each pointer written as a `$platform` or * `$generated` marker, into one JSON row. Returns `undefined` for a service * with no input schema. A service whose binding holds a secret pointer is * judged a second time, with a stand-in box in place of each secret pointer, @@ -583,19 +504,9 @@ export function serializeInput( } const { validated, generated, absent, resolved } = judged; refuseLostSecrets(node, address, resolved); - const emittedSecrets = new Set(); - const document = substitutePointers(validated, generated, address, emittedSecrets); - return { - key: inputKey(address), - value: JSON.stringify(document), - absent, - generated, - // The names the DOCUMENT actually carries (a schema transform can drop a - // bound leaf), sorted and unique: the fingerprint depends only on the set - // of referenced variables, so a binding refactor that reorders or - // duplicates leaves cannot move it. - secrets: [...emittedSecrets].sort(), - }; + const markers: PlatformMarkerUse[] = []; + const document = substitutePointers(validated, generated, address, markers); + return { key: inputKey(address), value: JSON.stringify(document), absent, generated, markers }; } /** @@ -737,7 +648,7 @@ export function inputValidationFailure( /** * Walks the VALIDATED output and writes each pointer as its document marker: - * an `envSecret` pointer as `{ "$secret": name }`, and a `generatedParam` or + * an `envSecret` pointer as a `$platform` marker with `secret: true`, and a `generatedParam` or * `serviceOrigin` pointer as `{ "$generated": VAR, "redacted": }`, found * by the input path its binding put it at. A pointer is recognised by its * brands, never by identity, so a copy the schema made still counts. Escapes @@ -749,15 +660,15 @@ function substitutePointers( value: unknown, generated: readonly GeneratedLeaf[], address: string, - emittedSecrets: Set, + markers: PlatformMarkerUse[], ): unknown { const generatedByPath = new Map(generated.map((leaf) => [leaf.path, leaf])); - const walk = (v: unknown, path: string): unknown => { + const walk = (v: unknown, path: string, inArray = false): unknown => { const leaf = generatedByPath.get(path); if (isSecretSource(v)) { const name = secretName(v, path === '' ? 'the input binding root' : `input key "${path}"`); - emittedSecrets.add(name); - return { [SECRET_MARKER]: name }; + markers.push({ name, path, secret: true, inArray }); + return platformMarker(name, true); } if (isParamSource(v)) { if (leaf === undefined) { @@ -766,7 +677,7 @@ function substitutePointers( 'binding did not put there: the schema moved it. A schema may check such a field, never move or transform it.', ); } - return { [GENERATED_MARKER]: leaf.varName, redacted: leaf.redacted }; + return generatedMarker(leaf.varName, leaf.redacted); } if (leaf !== undefined && v !== undefined) { throw new Error( @@ -781,13 +692,14 @@ function substitutePointers( "platform variable); bind the field with envSecret('NAME') instead.", ); } - if (Array.isArray(v)) return v.map((m, i) => walk(m, path === '' ? String(i) : `${path}.${i}`)); + if (Array.isArray(v)) { + return v.map((m, i) => walk(m, path === '' ? String(i) : `${path}.${i}`, true)); + } if (isPlainObject(v)) { const out: Record = {}; for (const [key, member] of Object.entries(v)) { if (member === undefined) continue; - const written = ESCAPABLE_KEY.test(key) ? `$${key}` : key; - out[written] = walk(member, path === '' ? key : `${path}.${key}`); + out[escapeKey(key)] = walk(member, path === '' ? key : `${path}.${key}`); } return out; } @@ -808,14 +720,6 @@ function substitutePointers( return walk(value, ''); } -/** - * Boot: read the one input document row (address-free after run()'s - * re-stash), swap each `$secret` pointer for a redacting box over the named - * platform var, unescape reserved keys, validate with the service's own - * schema, and return the typed object. A missing row is a loud failure naming - * the env var — a deployed environment always writes it; a local harness must - * supply it (set the row to the serialized input document). - */ export function readInput(node: ServiceNode, address: string): unknown { const schema = node.inputSchema; if (schema === undefined) { @@ -844,53 +748,18 @@ export function readInput(node: ServiceNode, address: string): unknown { `document (set ${key} to it).`, ); } - const hydrated = hydrateInputDocument(parsed, key); + const { value, unset } = readDocument(parsed, { + variable: (name) => process.env[name], + secret: (text) => new SecretBox(text), + }); try { - return standardValidateSync(schema, hydrated); + return standardValidateSync(schema, value); } catch (cause) { const message = cause instanceof Error ? cause.message : String(cause); - throw new Error(`invalid input document (env ${key}): ${message}`); - } -} - -/** Reverses the document encoding: `$secret` pointers become redacting boxes; a `$generated` pointer becomes a box iff its `redacted` facet is true, a plain string otherwise; escaped reserved keys drop one "$". */ -function hydrateInputDocument(value: unknown, key: string): unknown { - if (isWrittenSecretPointer(value)) { - const name = value[SECRET_MARKER]; - const secret = process.env[name]; - if (secret === undefined || secret === '') { - throw new Error( - `secret input is not provisioned (env ${key} → ${name}): the platform var "${name}" is unset or empty.`, - ); - } - return new SecretBox(secret); - } - if (isGeneratedPointer(value)) { - const varName = value[GENERATED_MARKER]; - const generated = process.env[varName]; - if (generated === undefined || generated === '') { - throw new Error( - `generated input is not provisioned (env ${key} → ${varName}): the platform variable ` + - `"${varName}" is unset or empty — the deploy provisions this variable, so its absence ` + - 'means the deploy and the running service disagree.', - ); - } - // Boot is schema-blind, so the redacted facet rides the pointer: box iff it - // says so, plain string otherwise. The schema field the author wrote (a - // `secretString()` box or a plain string) agrees with the facet by - // construction — a disagreement is a wiring error validation catches (§ 4.5). - return value.redacted ? new SecretBox(generated) : generated; - } - if (Array.isArray(value)) return value.map((member) => hydrateInputDocument(member, key)); - if (isPlainObject(value)) { - const out: Record = {}; - for (const [k, member] of Object.entries(value)) { - const read = ESCAPED_KEY.test(k) ? k.slice(1) : k; - out[read] = hydrateInputDocument(member, key); - } - return out; + throw new Error( + `The input of service "${node.name}" (environment variable ${key}) is invalid: ${withUnsetVariables(message, unset)}`, + ); } - return value; } // ——— Reserved provider params (ADR-0031): a provider-side minted value — diff --git a/packages/1-prisma-cloud/1-extensions/target/src/variable-name.ts b/packages/1-prisma-cloud/1-extensions/target/src/variable-name.ts new file mode 100644 index 000000000..3ed9e4415 --- /dev/null +++ b/packages/1-prisma-cloud/1-extensions/target/src/variable-name.ts @@ -0,0 +1,36 @@ +const VARIABLE_NAME = /^[A-Z][A-Z0-9_]*$/; + +const DATABASE_URL_REASON = + 'Prisma Cloud sets and manages DATABASE_URL and DATABASE_URL_POOLED itself. Declare the database your service uses and read its url from that connection.'; +const SHELL_REASON = + 'under prisma dev the local target sets PATH and HOME from your shell, so a value on the platform would never be seen locally.'; + +const RESERVED_NAMES: ReadonlyMap = new Map([ + ['DATABASE_URL', DATABASE_URL_REASON], + ['DATABASE_URL_POOLED', DATABASE_URL_REASON], + ['PORT', "boot sets PORT to the service's port."], + ['PATH', SHELL_REASON], + ['HOME', SHELL_REASON], +]); + +export function assertVariableName(name: string, factory: string): void { + if (typeof name !== 'string' || name.length === 0) { + throw new Error( + `${factory}() requires a non-empty platform variable name, for example ${factory}('API_KEY').`, + ); + } + if (!VARIABLE_NAME.test(name)) { + throw new Error( + `${factory} name "${name}" is not a platform variable name: it must be an upper-case letter followed by upper-case letters, digits and underscores, for example API_KEY.`, + ); + } + if (name.startsWith('COMPOSER_')) { + throw new Error( + `${factory} name "${name}" is reserved: names starting with COMPOSER_ are Composer's own platform variables.`, + ); + } + const reason = RESERVED_NAMES.get(name); + if (reason !== undefined) { + throw new Error(`${factory} name "${name}" is reserved: ${reason}`); + } +} diff --git a/packages/9-public/composer-prisma-cloud/package.json b/packages/9-public/composer-prisma-cloud/package.json index 67507687e..374ed8765 100644 --- a/packages/9-public/composer-prisma-cloud/package.json +++ b/packages/9-public/composer-prisma-cloud/package.json @@ -69,6 +69,7 @@ "@internal/email": "workspace:0.38.0", "@internal/foundation": "workspace:0.38.0", "@internal/http": "workspace:0.38.0", + "@internal/input-document": "workspace:0.38.0", "@internal/local-target": "workspace:0.38.0", "@internal/lowering": "workspace:0.38.0", "@internal/nextjs": "workspace:0.38.0", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 21328cc2d..c9dfe80d8 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -922,6 +922,21 @@ importers: specifier: ^6.0.3 version: 6.0.3 + packages/1-prisma-cloud/0-lowering/input-document: + devDependencies: + '@internal/tsdown-config': + specifier: workspace:0.38.0 + version: link:../../../0-framework/0-foundation/tsdown-config + '@types/bun': + specifier: ^1.3.13 + version: 1.3.14 + tsdown: + specifier: ^0.22.7 + version: 0.22.12(typescript@6.0.3) + typescript: + specifier: ^6.0.3 + version: 6.0.3 + packages/1-prisma-cloud/0-lowering/local-target: dependencies: '@alchemy.run/prisma': @@ -936,6 +951,9 @@ importers: '@internal/dev-emulators': specifier: workspace:0.38.0 version: link:../dev-emulators + '@internal/input-document': + specifier: workspace:0.38.0 + version: link:../input-document '@internal/lowering': specifier: workspace:0.38.0 version: link:../lowering @@ -1070,6 +1088,9 @@ importers: '@internal/http': specifier: workspace:0.38.0 version: link:../../../0-framework/2-authoring/http + '@internal/input-document': + specifier: workspace:0.38.0 + version: link:../../0-lowering/input-document '@internal/local-target': specifier: workspace:0.38.0 version: link:../../0-lowering/local-target @@ -1562,6 +1583,9 @@ importers: '@internal/http': specifier: workspace:0.38.0 version: link:../../0-framework/2-authoring/http + '@internal/input-document': + specifier: workspace:0.38.0 + version: link:../../1-prisma-cloud/0-lowering/input-document '@internal/local-target': specifier: workspace:0.38.0 version: link:../../1-prisma-cloud/0-lowering/local-target diff --git a/scripts/e2e-first-deploy.ts b/scripts/e2e-first-deploy.ts new file mode 100644 index 000000000..f436814bf --- /dev/null +++ b/scripts/e2e-first-deploy.ts @@ -0,0 +1,115 @@ +#!/usr/bin/env bun +import * as Effect from 'effect/Effect'; +import * as Layer from 'effect/Layer'; +import { + layer, + ManagementClient, +} from '../packages/1-prisma-cloud/0-lowering/lowering/src/client.ts'; +import { + isProductionEnvironment, + resolveContainer, +} from '../packages/1-prisma-cloud/0-lowering/lowering/src/container.ts'; +import { fromEnv } from '../packages/1-prisma-cloud/0-lowering/lowering/src/credentials.ts'; +import { call } from '../packages/1-prisma-cloud/0-lowering/lowering/src/http.ts'; + +const usage = + 'Usage: e2e-first-deploy.ts [ ...], run from the app directory, with STACK_NAME, ENVIRONMENT, PRISMA_WORKSPACE_ID, GITHUB_WORKSPACE and each NAME set'; + +function fail(message: string): never { + console.error(`::error::${message}`); + process.exit(1); +} + +function requiredEnv(name: string): string { + const value = process.env[name]; + if (value === undefined || value.length === 0) fail(`${name} is required. ${usage}`); + return value; +} + +const names = process.argv.slice(2); +if (names.length === 0) fail(usage); +const stackName = requiredEnv('STACK_NAME'); +const environment = requiredEnv('ENVIRONMENT'); +const workspaceId = requiredEnv('PRISMA_WORKSPACE_ID'); +const workspace = requiredEnv('GITHUB_WORKSPACE'); +const variables = names.map((key) => ({ key, value: requiredEnv(key) })); + +const isRecord = (value: unknown): value is Readonly> => + typeof value === 'object' && value !== null && !Array.isArray(value); + +function deployError(stdout: string): { readonly code: string; readonly summary: string } { + const last = stdout.trim().split('\n').at(-1) ?? ''; + let frame: unknown; + try { + frame = JSON.parse(last); + } catch { + fail(`The first deploy's last output line is not JSON: ${last}`); + } + const envelope = isRecord(frame) ? frame['envelope'] : undefined; + const error = isRecord(envelope) ? envelope['error'] : undefined; + if ( + !isRecord(error) || + typeof error['code'] !== 'string' || + typeof error['summary'] !== 'string' + ) { + fail(`The first deploy's result has no error: ${last}`); + } + return { code: error['code'], summary: error['summary'] }; +} + +const deploy = Bun.spawnSync( + [ + 'bun', + `${workspace}/node_modules/.bin/prisma`, + 'deploy', + '--name', + stackName, + '--environment', + environment, + '--json', + ], + { stdout: 'pipe', stderr: 'inherit' }, +); +const stdout = deploy.stdout.toString(); +console.log(stdout); + +if (deploy.exitCode === 0) { + fail(`The first deploy succeeded, but the new project has no value for: ${names.join(', ')}`); +} +const error = deployError(stdout); +if (error.code !== 'DEPLOY.PREFLIGHT_FAILED') { + fail(`The first deploy failed with ${error.code}, not DEPLOY.PREFLIGHT_FAILED`); +} +const wordsInError = new Set(error.summary.split(/[^A-Za-z0-9_]+/)); +for (const name of names) { + if (!wordsInError.has(name)) fail(`The first deploy stopped at preflight without naming ${name}`); +} + +const program = Effect.gen(function* () { + const { projectId, branchId } = yield* resolveContainer({ + workspaceId, + appName: stackName, + environment, + ensure: false, + }); + const scope = isProductionEnvironment(environment) + ? { class: 'production' as const } + : branchId === undefined + ? undefined + : { class: 'preview' as const, branchId }; + if (scope === undefined) { + return yield* Effect.fail( + new Error(`Project ${projectId} has no branch for the environment ${environment}.`), + ); + } + const client = yield* ManagementClient; + for (const { key, value } of variables) { + yield* call(() => + client.POST('/v1/environment-variables', { body: { projectId, ...scope, key, value } }), + ); + const where = 'branchId' in scope ? `preview, branch ${scope.branchId}` : 'production'; + console.log(`Set ${key} on project ${projectId} (${where}).`); + } +}); + +await Effect.runPromise(program.pipe(Effect.provide(layer().pipe(Layer.provideMerge(fromEnv()))))); diff --git a/scripts/e2e-generate-platform-variables.ts b/scripts/e2e-generate-platform-variables.ts new file mode 100644 index 000000000..043e9b529 --- /dev/null +++ b/scripts/e2e-generate-platform-variables.ts @@ -0,0 +1,19 @@ +#!/usr/bin/env bun +import { randomBytes } from 'node:crypto'; +import { appendFileSync } from 'node:fs'; + +const names = process.argv.slice(2); +const githubEnv = process.env['GITHUB_ENV']; +if (names.length === 0 || !githubEnv) { + console.error( + 'Usage: e2e-generate-platform-variables.ts [ ...], with GITHUB_ENV set', + ); + process.exit(1); +} + +for (const name of names) { + const value = randomBytes(32).toString('hex'); + console.log(`::add-mask::${value}`); + appendFileSync(githubEnv, `${name}=${value}\n`); + console.log(`Generated a random value for ${name}.`); +} diff --git a/skills/prisma-composer-core-concepts/SKILL.md b/skills/prisma-composer-core-concepts/SKILL.md index 37bc6c21b..759c2cf45 100644 --- a/skills/prisma-composer-core-concepts/SKILL.md +++ b/skills/prisma-composer-core-concepts/SKILL.md @@ -63,13 +63,7 @@ export default appRoot('store', ({ provision }) => { }); ``` -Because interfaces are typed, **the compiler verifies every wire**. A dependency -wired to the wrong producer, a missing RPC handler, a literal input value of -the wrong shape: all of it fails `tsc`, not the deploy. Env-bound input is -the exception: those values exist only at deploy, so secret-binding -mismatches and missing platform variables surface as early deploy-time -refusals instead (see Two channels below). Typecheck, then build, then -deploy; don't use the cloud to find out whether the wiring is correct. +Because interfaces are typed, **the compiler verifies every wire**. A dependency wired to the wrong producer, a missing RPC handler, a literal input value of the wrong shape: all of it fails `tsc`, not the deploy. Env-bound input is the exception: those values exist only at deploy, so secret-binding mismatches and missing platform variables a service needs surface as early deploy-time refusals instead (see Two channels below). Typecheck, then build, then deploy; don't use the cloud to find out whether the wiring is correct. Composer itself is target-agnostic: `@prisma/composer` carries authoring, testing, and the CLI, coupled to no platform. A deploy target is an extension @@ -142,12 +136,8 @@ platform variable and never carries the value. Rules that bite: 1. **Secretness is checked by the binding's type and by the deploy.** A `provision()` binding, a service's or a module's, is typed from the schema: a literal or a `SecretBox` where the schema expects `SecretString`, or `envSecret` in a text field, fails to compile, and the deploy refuses it where the schema has no types. An ArkType schema with a `SecretString` field cannot also transform its input: a morph anywhere in it (`'string.trim'`, `.pipe()`, `field(type('string.integer.parse'))`) or `'+': 'delete'` copies the input, the copied secret loses its value, and the deploy refuses it. Transform in the service code instead; a Zod schema keeps the box. -2. **`envParam` values arrive as raw strings**; bind them to string fields. A field of a narrower type, such as `email()`'s `deliveryMode`, refuses `envParam` at compile time; declare that field in the root's `input` instead, for `email()` from the exported `emailDeliveryModes`. In a service's `input`, the deploy reads the variable from the app configuration on every deploy and writes its value into the input document. As a service param, the deploy writes a pointer row to a platform variable, which the service reads at boot. `envSecret` always lives on the platform: each environment keeps its own copy, and the app secrets only seed a missing name (the deploy fails early, naming the variable, when both lack it). -3. **Absence is the schema's call.** An env-bound field whose variable is - unset or empty resolves to *key omitted*, which is legal only if the - schema allows it (optional field, union arm). The deploy report prints the - serialized input document (secrets ride as `{"$secret":"VAR"}` pointers) - and every key that resolved absent. +2. **`envParam` values arrive as raw strings**; bind them to string fields. A field of a narrower type, such as `email()`'s `deliveryMode`, refuses `envParam` at compile time; declare that field in the root's `input` instead, for `email()` from the exported `emailDeliveryModes`. In a service's `input`, the deploy reads the variable from the app configuration on every deploy and writes its value into the input document. `envParam()` in an input binding is deprecated: declare the value as a field of the root's `input` and pass it on. As a service param, `envParam` names a platform variable, which the service reads at boot. When that variable has no value, the param takes its default. An optional param with no default is left out. `envSecret` always lives on the platform. The operator sets it there with `prisma project env add 'NAME=' --project --role production`. Use `--role preview` for every preview environment, or `--role preview --branch ` for one. The deploy does not copy local values (`.env`, the shell) to platform variables. The deploy fails only when a variable that a service's schema requires has no value. A param with no default that is not optional is also required. The error names the variable and the command that sets it. The other variables with no value print under `Platform variables with no value:`. Prisma Cloud reports a service that exits at boot as running and promotes it; read the error with `prisma service logs `. +3. **The schema decides whether a field may be absent.** An env-bound field whose variable is unset or empty resolves to *key omitted*. That is allowed only if the schema allows it, for example with an optional field or a union member. An `envSecret()` field is left out at boot when its platform variable is unset or empty. The deploy report prints the serialized input document (each secret shown as `$platform VAR`) and every key that resolved absent. 4. **The reserved `port` is outside the schema.** Read it through `service.port()`, never `process.env`. The framework also exports `PORT` for Next.js standalone, which binds it itself. @@ -158,7 +148,8 @@ Rules that bite: refuse both, naming `input`. A service's `params` stay. 6. `input.apiKey.expose()` is the only way to a secret's value; the box redacts everywhere else (logs, JSON, errors). -7. **A field bound to `serviceOrigin()` or a plain `generatedParam()` is wrapped**: `field(type('string'))`. At deploy the schema sees a pointer, and a plain `'string'` field refuses it: a compile error in a service's or a module's binding, and a deploy error where the schema has no types. `service.input()` and `mockService`'s input double read the field as its inner type. +7. **Platform variable names are upper case**: `^[A-Z][A-Z0-9_]*$`. `COMPOSER_…`, `DATABASE_URL`, `DATABASE_URL_POOLED`, `PORT`, `PATH` and `HOME` are reserved, and one name cannot be both an `envParam()` and an `envSecret()`, or both an `envParam()` and a root input field. Each fails before anything is created. +8. **A field bound to `serviceOrigin()` or a plain `generatedParam()` is wrapped**: `field(type('string'))`. At deploy the schema sees a pointer, and a plain `'string'` field refuses it: a compile error in a service's or a module's binding, and a deploy error where the schema has no types. `service.input()` and `mockService`'s input double read the field as its inner type. ### Values that differ per environment @@ -166,7 +157,7 @@ The graph is the same in every environment; only configuration may differ. - A module declares `input` (one Standard Schema) as a service does; `provision(module, { input })` binds it, and the body receives `input`, a dictionary of values: it passes each field on by name into a binding or, for a plain field, lists it in `resolve()`. Passing the whole `input` on fails to compile, and the deploy refuses it. A root's `input` is the app configuration, and a module's `input` takes every source a service's does. - `field(inner)` from `@prisma/composer/arktype` or `@prisma/composer/zod` wraps one leaf field's schema (`field(type('string'))`, `field(z.string())`) so it also accepts a plain pointer, which is what a value known only at boot such as `serviceOrigin()` is at deploy. In the body the field is `Value>`: pass it into another wrapped field or a service's input, never a plain `string` field (compile error). Wrap leaves, not object schemas; an ArkType default goes outside: `field(type('string')).default('x')`. Other schema libraries: `Pointer`, `isPlainPointer()` and `isSecretPointer()` from `@prisma/composer`. Wrap every leaf of a reusable module's schema other than a `secretString()` field, unless a `resolve()` lists it; keep a root's fields plain. `field(secretString())` is wrong: it accepts a plain pointer into a secret field, and the service fails at boot. -- `composer({ config: [...], secrets: [...] })` lists config adapters; each gets the environment's name and returns values, later wins by top-level key. Both default to `[environmentVariables()]` (the shell); a list you write replaces the default. `dotenv()` reads `.env`, `.env.local`, `.env.`, `.env..local`, later wins. The app configuration is the root's input and what `envParam()` reads; pass the root's input fields on by name, as every module does; its whole `input` would be the whole app configuration, by default every shell variable. The app secrets only seed `envSecret()` platform variables and `prisma dev`'s secrets, never the root's input. Both import from `@prisma/composer/config`. Credentials (`PRISMA_SERVICE_TOKEN`) still come from the shell. `dotenv()` reads `.env`, `.env.local`, `.env.`, `.env..local` through `dotenv-flow` (no `.env.local` for `test`); remove `import 'dotenv/config'` when you list it. Composer redacts the values under the names an `envSecret()` reads, in either list; the lists' other values are not secrets and print as they are, apart from values that look like credentials. Under the CLI the deploy engine's own output is not redacted yet. +- `composer({ config: [...], secrets: [...] })` lists config adapters; each gets the environment's name and returns values, later wins by top-level key. Both default to `[environmentVariables()]` (the shell); a list you write replaces the default. `dotenv()` reads `.env`, `.env.local`, `.env.`, `.env..local`, later wins. The app configuration is the root's input and what `envParam()` reads; pass the root's input fields on by name, as every module does; its whole `input` would be the whole app configuration, by default every shell variable. The app secrets give `prisma dev` its secrets; a deploy reads them only to redact. It does not fill a platform variable from them, and never reads them as the root's input. Both import from `@prisma/composer/config`. Credentials (`PRISMA_SERVICE_TOKEN`) still come from the shell. `dotenv()` reads `.env`, `.env.local`, `.env.`, `.env..local` through `dotenv-flow` (no `.env.local` for `test`); remove `import 'dotenv/config'` when you list it. Composer redacts the values under the names an `envSecret()` reads, in either list; the lists' other values are not secrets and print as they are, apart from values that look like credentials. Under the CLI the deploy engine's own output is not redacted yet. - A field exists only under a plain object: a path into a string, an array or a secret, such as `input.tags[0]`, fails the deploy naming the path. - A truthiness check such as `if (input.enabled)` is not caught: the field is an object, so the branch always runs, in every environment. A module's structure cannot depend on its input; to provision something only sometimes, let the parent decide whether to provision it. - The module body receives `environment`; `environment.name` is a value standing for the name `--environment` gave (`production` for `prisma deploy`, `dev` for `prisma dev` by default). diff --git a/test/integration/test/fixtures/local-dev/module.ts b/test/integration/test/fixtures/local-dev/module.ts index ee0b70631..d353db28a 100644 --- a/test/integration/test/fixtures/local-dev/module.ts +++ b/test/integration/test/fixtures/local-dev/module.ts @@ -8,7 +8,7 @@ * required, so dev stops when `LOCALDEV_FIXTURE_API_KEY` is unset; * `webhookSecret` is optional, so an unset `LOCALDEV_FIXTURE_WEBHOOK_SECRET` * reaches the service as absent. The param rides the reserved `port` channel, - * bound to `LOCALDEV_FIXTURE_GREETING`; dev stops when it is unset. The names + * bound to `LOCALDEV_FIXTURE_GREETING`; `port` has a default, so dev starts when it is unset. The names * are fixture-private, chosen not to collide with any real platform env var. */ import { appRoot } from '@prisma/composer'; diff --git a/test/integration/test/local-dev-criteria-4-5.integration.ts b/test/integration/test/local-dev-criteria-4-5.integration.ts index a46aa1b0d..52ebc4053 100644 --- a/test/integration/test/local-dev-criteria-4-5.integration.ts +++ b/test/integration/test/local-dev-criteria-4-5.integration.ts @@ -15,8 +15,8 @@ * * Criterion 5 (value sourcing): every value comes from the environment of * the run, either the shell or a `.env` the fixture's `prisma.config.ts` - * loads. A required secret or env-sourced param that is unset or empty stops - * dev with an error naming it. An optional secret that is unset or empty + * loads. A required secret that is unset or empty stops dev with an error + * naming it. An optional secret that is unset or empty * reaches the service as absent. A value changed between two runs reaches * the running service on the next warm start. * @@ -278,7 +278,7 @@ async function expectPreflightFailure(env: NodeJS.ProcessEnv): Promise { function assertNamesMissing(log: string, variable: string, service: string): void { // In the structured frame the quotes around the service name arrive JSON-escaped. - const named = new RegExp(`${variable}\\s+\\(required by service \\\\?"${service}\\\\?"`); + const named = new RegExp(`${variable}\\s+\\(read by service \\\\?"${service}\\\\?"`); assert(named.test(log), `the error must name ${variable} and service "${service}"`); assert( log.includes('environmentVariables() reads the shell you run `prisma dev` from'), @@ -325,16 +325,6 @@ async function main(): Promise { } console.log('[proving] PASS: a required secret that is unset or empty stops dev, naming it'); - // ——— Criterion 5: a required env-sourced param that is unset stops dev ——— - const paramLog = await expectPreflightFailure( - baseEnv({ - LOCALDEV_FIXTURE_API_KEY: 'a-real-looking-key', - LOCALDEV_FIXTURE_GREETING: undefined, - }), - ); - assertNamesMissing(paramLog, 'LOCALDEV_FIXTURE_GREETING', 'localdevs4fixture.bkg'); - console.log('[proving] PASS: an unset env-sourced param stops dev, naming it'); - // ——— Criterion 5: a value loaded by prisma.config.ts reaches the service; an unset optional secret arrives absent ——— fs.writeFileSync(envFile, 'LOCALDEV_FIXTURE_API_KEY=from-the-config-file\n'); session = await startDevUntilReady( diff --git a/tsconfig.depcruise.json b/tsconfig.depcruise.json index 8e46820ff..d423cea87 100644 --- a/tsconfig.depcruise.json +++ b/tsconfig.depcruise.json @@ -90,6 +90,9 @@ "@internal/s3-protocol": [ "./packages/1-prisma-cloud/0-lowering/s3-protocol/src/exports/index.ts" ], + "@internal/input-document": [ + "./packages/1-prisma-cloud/0-lowering/input-document/src/exports/index.ts" + ], "@internal/dev-emulators/compute-main": [ "./packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/compute-main.ts" ],