From b7826301f912a2ef851474a063fd6cfc38ae0fe7 Mon Sep 17 00:00:00 2001 From: David Gamero Date: Fri, 7 Aug 2026 16:29:10 -0400 Subject: [PATCH 1/4] fix(kickstart): build with az acr build and cover all AKS deployment safeguards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Kickstart runs in Azure Cloud Shell, which has no Docker daemon, but the phase skills assumed a local one. Generate listed `docker build` first with ACR as a fallback, and its image-content check (`docker run ... ls`) was local-only while the exit criteria still demanded the entry point be confirmed present. Review then made that same unrunnable check a hard FAIL, so in Cloud Shell the agent would either stall or fabricate a pass. Images are now built with `az acr build` unconditionally — server-side on the ACR remote task builders, never `docker build`. One build path keeps the validated image and the deployed image identical. Entry-point verification is a build-time `RUN test -f ` assertion in the Dockerfile, which fails the ACR build directly and works before the cluster exists. Also documents the Cloud Shell specifics that still matter: clouddrive clone target, AcrPush requirement, context upload cost, idle-session handling. Separately, the safeguard checklist was a pod-security rule set, not the AKS Deployment Safeguards policy set — 9 of the 10 policies were unrepresented, including the mutating ones (resource requests, anti-affinity/topology spread, reserved system pool taints). Omitting those doesn't block admission; the cluster rewrites the object, so the deployed state silently stops matching the generated YAML. The checklist is now split into Part A (the 10 policies, with mutation outcomes) and Part B (pod security), Generate emits compliant manifests up front, and Deploy gains a `safeguard` failure class plus a server-side dry-run preview. --- agents/kickstart-reviewer.agent.md | 4 +- agents/kickstart.agent.md | 8 +- skills/kickstart-deploy/SKILL.md | 7 +- skills/kickstart-discover/SKILL.md | 12 +- skills/kickstart-generate/SKILL.md | 54 +++++++-- skills/kickstart-review/SKILL.md | 8 +- skills/kickstart-safeguard-checklist/SKILL.md | 109 ++++++++++++++++-- 7 files changed, 172 insertions(+), 30 deletions(-) diff --git a/agents/kickstart-reviewer.agent.md b/agents/kickstart-reviewer.agent.md index 0b2fdefaf..10414f3b9 100644 --- a/agents/kickstart-reviewer.agent.md +++ b/agents/kickstart-reviewer.agent.md @@ -19,7 +19,7 @@ You review deployment artifacts generated by the Kickstart agent. Your job is to ## Review Process 1. Invoke `/kickstart-review` for the full per-artifact checklist (Dockerfile, Kubernetes manifests, Bicep, GitHub Actions) plus dry-run validation (`kubectl apply --dry-run=client`, `az bicep build`). -2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. +2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. Report **Part A** (policy-enforced AKS Deployment Safeguards) separately from **Part B** (pod security / best practice), and treat a FAIL on a *mutating* safeguard (A2 requests, A3 spread/anti-affinity, A6 reserved taints) as a real defect — the cluster will rewrite the object rather than reject it, so it must be fixed in the YAML. 3. Invoke `/kickstart-security-hardening` for security-specific checks (RBAC scope, Key Vault, encryption, network isolation). Work through every generated artifact against those checklists — do not re-derive the criteria here. @@ -27,3 +27,5 @@ Work through every generated artifact against those checklists — do not re-der ## Output Present findings as a checklist with **PASS** ✓, **FAIL** ✗, or **WARN** ⚠ for each item. If any FAIL items exist, list specific fixes needed. + +Images are always built with `az acr build`, never `docker build` — never ask for local Docker output or tell the user to install Docker. Verify the entry point via the Dockerfile's build-time `RUN test -f ` assertion. diff --git a/agents/kickstart.agent.md b/agents/kickstart.agent.md index 958bae773..211ca37b8 100644 --- a/agents/kickstart.agent.md +++ b/agents/kickstart.agent.md @@ -66,7 +66,7 @@ Handle each starting point (ask via `vscode_askQuestions` only when it wasn't al }] } ``` - Clone with `run_in_terminal`, then load `/kickstart-samples` for the pre-filled profile and confirm it with the user. **Skip Phase 1's questions** — go straight to **Phase 2 (Configure Infrastructure)**. Do NOT ask the user for app name, port, language, or any discovery questions — but still run the quick structure scan from `/kickstart-samples` to confirm each service's build context, Dockerfile path, and entry point before generating anything. + Clone with `run_in_terminal`, then load `/kickstart-samples` for the pre-filled profile and confirm it with the user. **Skip Phase 1's questions** — go straight to **Phase 2 (Configure Infrastructure)**. Do NOT ask the user for app name, port, language, or any discovery questions — but still run the quick structure scan from `/kickstart-samples` to confirm each service's build context, Dockerfile path, and entry point, before generating anything. - **Use my current workspace**: Proceed to **Phase 1 (Discover)**. ## Phases @@ -85,15 +85,15 @@ Follow `/kickstart-configure-infra`. Do NOT pick subscriptions or run `az aks cr Follow `/kickstart-design`. Present architecture summary (container strategy, AKS Automatic, Gateway API, Workload Identity, ACR, monitoring). Get user approval via `vscode_askQuestions`. Run `/kickstart-cluster-status` before transitioning. ### 4 — Generate -Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Build and inspect each image (confirm the entry point landed) before exiting. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Emit manifests that already satisfy the mutating AKS Deployment Safeguards (CPU/memory **requests**, `topologySpreadConstraints` or pod anti-affinity, unique per-service Service selectors, probes, CSI `storageClassName`, no `kubernetes.azure.com/*` labels or `CriticalAddonsOnly` toleration). Build each image with `az acr build` (server-side; never `docker build`) and assert the entry point with a `RUN test -f` line in the Dockerfile before exiting. Run `/kickstart-cluster-status` before transitioning. ### 5 — Review -Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds with the entry point present, then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds with the entry point present via `az acr build` plus its build-time entry-point assertion — then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. ### 6 — Pre-Deploy Check Follow `/kickstart-handoff` — it carries the full strict-order playbook: cluster readiness (6a), metadata detection (6b), ACR attachment verification (6c — idempotent; the registry is usually already attached during cluster setup), kubelogin (6d), and the consolidated permission probes (6e–6g) via the bundled `aks.checkDeploymentPermissions` command. Escalate through `/kickstart-pim-activation` whenever a role assignment returns 403. Confirm readiness with the user via `vscode_askQuestions` before deploying. ### 7 — Deploy -Follow `/kickstart-deploy` — build & push to ACR (using each service's build context and Dockerfile path from the structure map, never `.`), get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. +Follow `/kickstart-deploy` — build & push to ACR with `az acr build` (server-side, never `docker build`; using each service's build context and Dockerfile path from the structure map, never `.`), get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. Once the app is running, offer to commit the generated artifacts (Dockerfile, `k8s/`, `infra/`, workflow). If the user wants to commit or open a PR, follow `/kickstart-github-pr-conventions` for branch naming, Conventional Commits, and PR structure. diff --git a/skills/kickstart-deploy/SKILL.md b/skills/kickstart-deploy/SKILL.md index 69405e518..fdc6c5f77 100644 --- a/skills/kickstart-deploy/SKILL.md +++ b/skills/kickstart-deploy/SKILL.md @@ -11,12 +11,16 @@ Deploy using Azure CLI and `kubectl`. Execute each step via `run_in_terminal`, c ## Steps 1. **Build and push**: `az acr build --registry --image : -f ` + `az acr build` runs **server-side on the ACR remote task builders** — always use it, never `docker build` + `docker push`. Kickstart runs in Azure Cloud Shell, which has no Docker daemon, and a single remote build keeps the pushed digest and the deployed image identical. Use the build context and Dockerfile path from the structure map — never assume repo root (`.`). For monorepos, build each service from its own context. Tag with a version (e.g. v1.0.0), never `:latest`. + Requires `AcrPush` or `Container Registry Tasks Contributor` (verified in Pre-Deploy Check). Keep the command in the foreground so its streamed log holds an idle Cloud Shell session open, and make sure `.dockerignore` excludes bulk (the whole context is uploaded on each build). + Third-party images the app depends on (Redis, Postgres, RabbitMQ) can't be `az acr build`-ed — bring them in with `az acr import --name --source docker.io/library/: --image :`. 2. **Get credentials**: `az aks get-credentials --resource-group --name --overwrite-existing` kubelogin handles AAD auth automatically (verified in Pre-Deploy Check). Never use `--admin`. -3. **Apply manifests**: `kubectl apply -f k8s/` +3. **Apply manifests**: preview with `kubectl apply --dry-run=server -f k8s/`, then `kubectl apply -f k8s/`. + Expect mutation: AKS Automatic's Deployment Safeguards rewrite on admission — missing CPU/memory requests get defaults, missing spread rules get anti-affinity/topology constraints, a stray `CriticalAddonsOnly` toleration is stripped. If Phase 4 generated compliant manifests these are no-ops; if `kubectl get -o yaml` differs from your file, that's the mutator, not a bug. 4. **Verify**: `kubectl get pods -n ` and `kubectl get services -n ` If pods not Ready, run `kubectl describe pod ` and `kubectl logs ` to diagnose. @@ -33,6 +37,7 @@ Classify failures: - **config** — missing or invalid configuration (subscription, RG, cluster, ACR, manifest) - **dependency** — missing CLI tool, extension, or container image - **cluster** — pod CrashLoopBackOff, ImagePullBackOff, scheduling, quota +- **safeguard** — admission rejected by a Deployment Safeguard (e.g. `:latest` tag, non-CSI StorageClass, duplicate Service selector, AKS-reserved label, node edit). Fix the manifest to satisfy the rule; never disable the policy. Provide specific `az` or `kubectl` fix commands. Offer retry via `vscode_askQuestions`. diff --git a/skills/kickstart-discover/SKILL.md b/skills/kickstart-discover/SKILL.md index 90bd0c36f..16669a98e 100644 --- a/skills/kickstart-discover/SKILL.md +++ b/skills/kickstart-discover/SKILL.md @@ -26,6 +26,16 @@ Never assume a flat repo. Apps often live in nested or monorepo layouts (`src/ } + ``` +- **Readiness *and* liveness probes** on every container — use the real health path and port from the structure map, never a guessed `/healthz`. +- **Unique Service selectors** — each Service must select exactly one workload. In a monorepo do **not** reuse `app: ` across services; use `app: ` (or `app.kubernetes.io/name` + `app.kubernetes.io/component`) so no two Services overlap. +- **No AKS-specific labels** — never set `kubernetes.azure.com/*` labels on your own objects. (`azure.workload.identity/*` labels and annotations are fine and required.) +- **No reserved system-pool taints/tolerations** — do not add a `CriticalAddonsOnly` toleration to app pods; it would place them on the system pool and AKS strips it anyway. +- **CSI StorageClass for any PVC** — set `storageClassName` explicitly to a CSI class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`); never rely on an in-tree or unset default. +- **Pinned image tags** — no `:latest` anywhere, including init containers and sidecars. +- **Allowed images only** — every image must come from the Phase 2 ACR (`.azurecr.io/...`) if the cluster restricts registries; flag any third-party image (Redis, Postgres, RabbitMQ) that would need importing via `az acr import`. +- **Never edit individual nodes** — no node-targeted manifests, `kubectl label node`, `kubectl taint node`, or node-name `nodeSelector`. Use node pools instead. **Bicep** (`infra/main.bicep`): AKS Automatic + ACR + Managed Identity + federated credential. Parameterized, pinned API versions. ARM resource IDs follow `/subscriptions/{sub}/resourceGroups/{rg}/providers/{ns}/{type}/{name}`. See `/kickstart-bicep-authoring` and `/kickstart-acr-integration`. @@ -37,18 +65,26 @@ Load these for detailed patterns as you author each artifact: - Use actual resource names from the Configure phase. - Never use `:latest` tags. - Honor each service's build context and entry point from the structure map; reuse existing Dockerfiles instead of duplicating them. -- All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath). +- All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath) **and** must pre-satisfy the mutating safeguards above so the cluster doesn't rewrite them on apply. +- Build images with `az acr build` only — never `docker build`. - After writing all files, confirm with user via `vscode_askQuestions`. ## Validate the build (before exit) -Do not hand off unbuilt artifacts. For each Dockerfile, build and inspect before announcing completion: +Do not hand off unbuilt artifacts. Build every Dockerfile with **`az acr build`** — server-side on the ACR remote task builders. Never `docker build`; there is no Docker daemon in Cloud Shell, and one build path keeps the validated image and the deployed image identical. -1. **Build** from the service's build context: - - Local Docker/Podman daemon available: `docker build -t kickstart-validate-:check -f `. - - Otherwise build in ACR (also catches missing `COPY` sources): `az acr build --registry --image kickstart-validate/:check -f `. -2. **Inspect contents** (when built locally): `docker run --rm kickstart-validate-:check ls -la ` — confirm the entry point and expected files landed where the app runs from. A build that succeeds but places files in the wrong dir is exactly the failure this step catches. -3. If the build fails or the entry point is missing, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. +1. **Build in ACR** from the service's own build context. This also catches missing `COPY`/`ADD` sources: + ```bash + az acr build --registry --image kickstart-validate/:check -f + ``` +2. **Assert the entry point at build time.** Add to the Dockerfile's final stage so a wrong path fails the ACR build itself: + ```dockerfile + RUN test -f + ``` + The assertion runs before the cluster exists, so a wrong path is caught immediately. +3. If the build fails or the assertion trips, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. + +Keep `.dockerignore` tight: the entire build context is uploaded to ACR on every build. ## Exit Criteria -All artifacts written, every Dockerfile builds, and the entry point is confirmed present in the image. Announce: "Artifacts generated and build-validated — moving to Review." +All artifacts written, every Dockerfile builds via `az acr build`, and the build-time entry-point assertion passes. Announce: "Artifacts generated and build-validated — moving to Review." diff --git a/skills/kickstart-review/SKILL.md b/skills/kickstart-review/SKILL.md index 9ddfffaf7..66f634636 100644 --- a/skills/kickstart-review/SKILL.md +++ b/skills/kickstart-review/SKILL.md @@ -12,7 +12,9 @@ Validate every artifact against security, correctness, and AKS Automatic complia **Dockerfile**: Multi-stage build, pinned base image, non-root user, `.dockerignore` present. Build context + every `COPY`/`ADD` source→destination resolves to real files; `CMD`/`ENTRYPOINT` runs the actual entry point; the image builds and the entry point is present in the built image. -**K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. +**Dockerfile build**: built with `az acr build` (never `docker build`), with a `RUN test -f ` assertion in the final stage. + +**K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, `topologySpreadConstraints` or anti-affinity, unique per-Service selectors, CSI `storageClassName` on any PVC, no `kubernetes.azure.com/*` labels, no `CriticalAddonsOnly` toleration, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. Run the full `/kickstart-safeguard-checklist` — the mutating safeguards there must already be satisfied in the YAML, not left to the cluster. **Bicep**: API versions pinned, parameterized env values, secure defaults (TLS 1.2+), outputs defined. @@ -26,13 +28,15 @@ Validate every artifact against security, correctness, and AKS Automatic complia |---|---|---|---|---| | `src/order-service` | `package.json`, `src/` | `/app` | `/app/server.js` | 3000 | - Then confirm (or run) the build validation from `/kickstart-generate` — the image must build and `ls ` must show the entry point. A missing or mismatched path is a FAIL. + Then confirm (or run) the build validation from `/kickstart-generate`: the image must build via `az acr build`, and the Dockerfile's build-time `RUN test -f ` assertion must be present and passing. A missing assertion or a failed build is a FAIL. Do not ask for `docker build` / `docker run` output — images are always built server-side in ACR. + 2. Run `/kickstart-safeguard-checklist` for the full safeguard rule set. 3. Run validation via `run_in_terminal`: ```bash kubectl apply --dry-run=client -f k8s/ az bicep build --file infra/main.bicep ``` + `--dry-run=server` is preferable when the cluster is reachable: it runs the admission webhooks, so it surfaces real Deployment Safeguard violations and shows the mutated result. 4. Present results as PASS ✓ / FAIL ✗ / WARN ⚠ per item. 5. If FAILs: use `vscode_askQuestions` — fix automatically (recommended), show details, or skip. 6. If WARNs only: confirm proceeding via `vscode_askQuestions`. diff --git a/skills/kickstart-safeguard-checklist/SKILL.md b/skills/kickstart-safeguard-checklist/SKILL.md index 40bb2b830..4b00dd0d4 100644 --- a/skills/kickstart-safeguard-checklist/SKILL.md +++ b/skills/kickstart-safeguard-checklist/SKILL.md @@ -6,9 +6,76 @@ disable-model-invocation: true # AKS Deployment Safeguard Checklist -This skill provides a comprehensive checklist for validating generated Kubernetes manifests against AKS security and deployment best practices. Use this during the Review phase to ensure all generated configurations comply with organizational policies. +This skill provides a comprehensive checklist for validating generated Kubernetes manifests. Use this during the Review phase. -## Safeguard Rules +It has **two parts**, and they are not interchangeable: + +- **Part A — AKS Deployment Safeguards.** The policies AKS Automatic actually enforces at admission. Some **mutate** your object instead of rejecting it, so a manifest that omits them will be silently rewritten by the cluster. Generate these correctly up front (see `/kickstart-generate`) so the mutators are no-ops. +- **Part B — Pod security & deployment best practice.** Additional hardening checks Kickstart applies. Valuable, but *not* the AKS Deployment Safeguards policy set — don't conflate the two when reporting. + +--- + +# Part A — AKS Deployment Safeguards (policy-enforced) + +| # | Safeguard policy | Severity | Mutation outcome if available | +|---|---|---|---| +| A1 | Cannot Edit Individual Nodes | HIGH | N/A — rejected, not mutated | +| A2 | Containers CPU and memory resource **requests** must be defined | HIGH | **Mutates** — sets default CPU/memory requests and enforces minimums | +| A3 | Must have anti-affinity rules or `topologySpreadConstraints` set | MEDIUM | **Mutates** — adds pod anti-affinity + topology spread constraints | +| A4 | No AKS-specific labels | MEDIUM | N/A | +| A5 | Containers should only use allowed images | HIGH | N/A | +| A6 | Reserved system pool taints | MEDIUM | **Mutates** — removes the `CriticalAddonsOnly` taint/toleration from user node pool workloads | +| A7 | Containers have readiness or liveness probes configured | HIGH | N/A | +| A8 | Clusters should use CSI driver StorageClass | MEDIUM | N/A | +| A9 | Services should use unique selectors | HIGH | N/A | +| A10 | Container images should not include `latest` tag | HIGH | N/A | + +### A1: cannot-edit-individual-nodes +- **Check**: No manifest, script, or command targets an individual Node — no `kind: Node` objects, no `kubectl label/taint/cordon node`, no `nodeSelector`/`nodeName` pinning to a specific node name. Use node pools and pool-level labels instead. +- [ ] Pass / Fail + +### A2: require-requests *(mutating)* +- **Check**: Every container (including init containers and sidecars) declares `resources.requests.cpu` **and** `resources.requests.memory`. Limits alone are not sufficient. +- **If omitted**: AKS injects defaults and raises anything below the enforced minimum — your applied object will differ from your YAML. +- [ ] Pass / Fail + +### A3: require-spread-or-anti-affinity *(mutating)* +- **Check**: Each Deployment/StatefulSet sets `spec.template.spec.topologySpreadConstraints` **or** `affinity.podAntiAffinity`. Prefer topology spread on `kubernetes.io/hostname` with `whenUnsatisfiable: ScheduleAnyway`. +- **If omitted**: AKS adds its own anti-affinity and spread constraints, which can change scheduling behavior you didn't plan for. +- [ ] Pass / Fail + +### A4: no-aks-specific-labels +- **Check**: No object sets a `kubernetes.azure.com/*` label. These are reserved for AKS. (`azure.workload.identity/*` labels and annotations are **not** covered by this rule and are required for Workload Identity.) +- [ ] Pass / Fail + +### A5: allowed-images-only +- **Check**: Every image resolves to a registry the cluster permits — normally the Phase 2 ACR (`.azurecr.io/...`). Flag any third-party image (`docker.io/...`, `ghcr.io/...`) that must first be brought in with `az acr import`. +- [ ] Pass / Fail + +### A6: reserved-system-pool-taints *(mutating)* +- **Check**: No app workload declares a `CriticalAddonsOnly` toleration, and no user node pool config sets that taint. AKS uses it to keep customer pods off the system pool. +- **If present**: AKS removes it, so any scheduling you based on it will not hold. +- [ ] Pass / Fail + +### A7: require-probes +- **Check**: Every container defines a `readinessProbe` **or** `livenessProbe` (Kickstart generates both). The path and port must match the app's real health endpoint from the structure map — not a guessed `/healthz`. +- [ ] Pass / Fail + +### A8: csi-storageclass +- **Check**: Every PersistentVolumeClaim sets `storageClassName` to a CSI-backed class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`). No in-tree provisioners, no reliance on an unset default. +- [ ] Pass / Fail (N/A when no PVCs) + +### A9: unique-service-selectors +- **Check**: No two Services share a selector, and each Service's selector matches exactly one workload. In monorepos do not reuse `app: ` across services — use `app: ` or `app.kubernetes.io/name` + `app.kubernetes.io/component`. +- [ ] Pass / Fail + +### A10: no-latest-tag +- **Check**: No image reference ends in `:latest` or omits a tag (an untagged image resolves to `latest`). Applies to init containers and sidecars too. +- [ ] Pass / Fail + +--- + +# Part B — Pod security & best practice ### Rule: no-privileged - **Severity**: HIGH @@ -18,7 +85,7 @@ This skill provides a comprehensive checklist for validating generated Kubernete ### Rule: require-limits - **Severity**: MEDIUM -- **Description**: All containers must declare resource limits (CPU and memory). +- **Description**: All containers must declare resource limits (CPU and memory). Requests are covered separately by the policy-enforced **A2**. - **Check**: Verify that `spec.containers[*].resources.limits` is defined for all containers - [ ] Pass / Fail @@ -28,12 +95,6 @@ This skill provides a comprehensive checklist for validating generated Kubernete - **Check**: Verify that `spec.volumes[*].hostPath` is null or not present - [ ] Pass / Fail -### Rule: no-latest-tag -- **Severity**: HIGH -- **Description**: Container images must not use the ':latest' tag. -- **Check**: Verify that `spec.containers[*].image` does not end with `:latest` -- [ ] Pass / Fail - ### Rule: no-privilege-escalation - **Severity**: HIGH - **Description**: Containers must not allow privilege escalation. @@ -111,13 +172,22 @@ This skill provides a comprehensive checklist for validating generated Kubernete When possible, use the `runCommands` tool to validate manifests programmatically: ```bash -# Dry-run validation against K8s API schemas +# Schema-only validation (works with no cluster) kubectl apply --dry-run=client -f k8s/ +# BEST: server dry-run runs the real admission webhooks — this is what actually +# evaluates Part A, and it shows you the mutated result before you commit to it. +kubectl apply --dry-run=server -f k8s/ + +# Diff your YAML against what the cluster would store (reveals safeguard mutations) +kubectl diff -f k8s/ + # Validate with kubeconform (if installed) kubeconform -strict -summary k8s/*.yaml ``` +`--dry-run=server` requires cluster credentials (Phase 6). Before that, evaluate Part A by reading the manifests. + ## Review Instructions When reviewing manifests, use this checklist to validate each safeguard rule: @@ -130,16 +200,31 @@ When reviewing manifests, use this checklist to validate each safeguard rule: 3. **Report results** in a summary table showing rule ID, status, and any notes 4. **Block on failures**: Any FAIL on a **HIGH-severity** rule must be fixed before the manifest proceeds to deployment 5. **Address medium-severity failures**: MEDIUM-severity FAILs should be resolved or explicitly justified before proceeding +6. **Never leave a mutating safeguard (A2, A3, A6) to the cluster.** A FAIL there won't block admission — AKS will quietly rewrite the object, so the deployed state stops matching the generated YAML and later `kubectl diff` output becomes confusing. Fix these in the manifest even though they "would work anyway." +7. **Report Part A and Part B separately** so the user can see policy compliance distinctly from hardening advice. ## Example Review Output +**Part A — AKS Deployment Safeguards** + +``` +| ID | Safeguard | Severity | Mutating | Status | Notes | +|----|-----------|----------|----------|--------|-------| +| A2 | require-requests | HIGH | yes | ✓ PASS | cpu 100m / memory 128Mi on all containers | +| A3 | spread-or-anti-affinity | MEDIUM | yes | ✓ PASS | topologySpreadConstraints on hostname | +| A9 | unique-service-selectors | HIGH | no | ✗ FAIL | api and worker Services both select app: store | +| A10 | no-latest-tag | HIGH | no | ✓ PASS | pinned tag v1.2.3 | +| ... | ... | ... | ... | ... | ... | +``` + +**Part B — Pod security & best practice** + ``` | Rule ID | Severity | Status | Notes | |---------|----------|--------|-------| | no-privileged | HIGH | ✓ PASS | securityContext.privileged is false | | require-limits | MEDIUM | ✓ PASS | All containers have CPU/memory limits | | no-hostpath | HIGH | ✓ PASS | No hostPath volumes defined | -| no-latest-tag | HIGH | ✓ PASS | Image uses pinned tag v1.2.3 | | ... | ... | ... | ... | ``` @@ -152,4 +237,4 @@ When reviewing manifests, use this checklist to validate each safeguard rule: --- -*Last updated: Safeguards from `packages/pack-aks-automatic/src/safeguards.json`* +*Part A: AKS Automatic Deployment Safeguards policy set (mutation outcomes per AKS docs). Part B: `packages/pack-aks-automatic/src/safeguards.json`.* From a2d02670a10276cbdc1f584f3d60ead7d715fe65 Mon Sep 17 00:00:00 2001 From: David Gamero Date: Thu, 20 Aug 2026 17:15:00 -0400 Subject: [PATCH 2/4] docs(kickstart): correct safeguard details and stamp generated resources Three corrections after checking the AKS Deployment Safeguards docs against what the checklist claimed: - The anti-affinity safeguard is replica-conditional. Its error names two replicas, so single-replica workloads aren't flagged. Also documents what the mutator actually injects (preferred anti-affinity weight 100, maxSkew 1 on kubernetes.io/hostname) and its selector-label priority. - The resource-requests mutator defaults are much larger than assumed: 500m CPU and 2048Mi memory per container, not a token value. Minimums are 100m / 100Mi. Worth stating plainly, since omitting requests on a multi-container pod is a real scheduling and cost surprise. - The CSI safeguard evaluates the StorageClass provisioner, not its name. In-tree kubernetes.io/azure-disk and azure-file are rejected in favour of disk.csi.azure.com and file.csi.azure.com. Also records that Gatekeeper runs fail-open, so safeguards are a backstop rather than a guarantee, and that the policy set is all-or-nothing. Adds provenance markers to generated artifacts. Nothing kickstart produced was identifiable as kickstart-generated, while sibling features in this extension already stamp theirs. Kubernetes objects now get an app.kubernetes.io/managed-by label plus a kickstart.aks.azure.com/version annotation, Dockerfiles get com.azure.aks.kickstart.* LABELs, and Bicep gets tags. The label/annotation split is deliberate. managed-by is selectable identity, so it belongs in a label; version is read rather than selected, and label values reject '+' so a semver with build metadata could not be stored in one. Both go on the object's own metadata rather than the pod template, since anything on the template changes the pod-template hash and would force a rollout on every version bump, and neither may enter a selector. The restricted-labels safeguard only inspects labels, so the version annotation is outside its scope, but the reserved kubernetes.azure.com prefix is still avoided on both fields. Version is omitted rather than guessed when it can't be determined; the skills are static markdown with no interpolation, so there is not yet a path for injecting the extension version. --- skills/kickstart-generate/SKILL.md | 40 +++++++++++++++++++ skills/kickstart-review/SKILL.md | 2 + skills/kickstart-safeguard-checklist/SKILL.md | 22 +++++++--- 3 files changed, 59 insertions(+), 5 deletions(-) diff --git a/skills/kickstart-generate/SKILL.md b/skills/kickstart-generate/SKILL.md index 2f4a322a0..18faa9228 100644 --- a/skills/kickstart-generate/SKILL.md +++ b/skills/kickstart-generate/SKILL.md @@ -29,6 +29,45 @@ Load these for detailed patterns as you author each artifact: **K8s Manifests** (`k8s/`): `namespace.yaml`, `deployment.yaml` (resource requests **and** limits, probes, `runAsNonRoot`, Workload Identity labels, env from ConfigMap/Secret), `service.yaml` (ClusterIP), `httproute.yaml` (Gateway API, not Ingress). See `/kickstart-workload-identity`. +### Stamp generated resources with kickstart provenance + +Every resource kickstart generates carries a marker so it can be identified later — both by a human reading the repo and by `kubectl` queries against a live cluster. + +**Kubernetes objects** — a label for identity, an annotation for the version: + +```yaml +metadata: + labels: + app.kubernetes.io/managed-by: aks-kickstart + annotations: + kickstart.aks.azure.com/version: "" +``` + +Rules for placement, which matter more than the choice of field: + +- Put both on the **object's own `metadata`** (Deployment, Service, HTTPRoute, ServiceAccount, ConfigMap...), **not** on `spec.template.metadata`. Anything on the pod template — label or annotation — changes the pod-template hash and forces a full rollout on the next version bump. (`kubectl rollout restart` works precisely by writing a pod-template annotation.) +- Never put either in `spec.selector` or in a Service selector. `spec.selector` is immutable after creation, and a version in a selector would make the Deployment un-updatable. It also collides with the unique-selector safeguard (A9). +- `app.kubernetes.io/managed-by` is the convention already used elsewhere in this extension, and its neutral prefix keeps it clear of the reserved `kubernetes.azure.com/*` namespace blocked by safeguard A4. The version is an **annotation** rather than a label because label values are capped at 63 chars and reject `+`, so a semver with build metadata (`2.4.0+abc123`) would be an invalid label value but is a perfectly fine annotation. + +**Dockerfile** — OCI labels in the final stage. Use a kickstart-specific key for the generator version; `org.opencontainers.image.version` means the *application's* version, not the tool's: + +```dockerfile +LABEL org.opencontainers.image.source="" \ + com.azure.aks.kickstart.generated="true" \ + com.azure.aks.kickstart.version="" +``` + +**Bicep** — Azure resources take `tags`: + +```bicep +tags: { + 'managed-by': 'aks-kickstart' + 'kickstart-version': '' +} +``` + +If you can't determine the extension version, emit the `managed-by` label / `generated` LABEL / `managed-by` tag and **omit the version entirely** — a wrong version is worse than an absent one, since it misattributes which release produced the artifact. Never guess or hardcode a version number. + ### Generate safeguard-compliant manifests up front AKS Automatic enforces **Deployment Safeguards**, and several of them *mutate* your manifest on apply. If you don't emit these yourself, the cluster silently rewrites the object and the Phase 7 `kubectl get`/`diff` output won't match what you generated. Emit all of the following by default — see `/kickstart-safeguard-checklist` for the authoritative rule list and mutation outcomes. @@ -65,6 +104,7 @@ AKS Automatic enforces **Deployment Safeguards**, and several of them *mutate* y - Use actual resource names from the Configure phase. - Never use `:latest` tags. - Honor each service's build context and entry point from the structure map; reuse existing Dockerfiles instead of duplicating them. +- Stamp every generated resource with the kickstart provenance marker above (object metadata only — never the pod template or a selector). - All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath) **and** must pre-satisfy the mutating safeguards above so the cluster doesn't rewrite them on apply. - Build images with `az acr build` only — never `docker build`. - After writing all files, confirm with user via `vscode_askQuestions`. diff --git a/skills/kickstart-review/SKILL.md b/skills/kickstart-review/SKILL.md index 66f634636..17874a433 100644 --- a/skills/kickstart-review/SKILL.md +++ b/skills/kickstart-review/SKILL.md @@ -16,6 +16,8 @@ Validate every artifact against security, correctness, and AKS Automatic complia **K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, `topologySpreadConstraints` or anti-affinity, unique per-Service selectors, CSI `storageClassName` on any PVC, no `kubernetes.azure.com/*` labels, no `CriticalAddonsOnly` toleration, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. Run the full `/kickstart-safeguard-checklist` — the mutating safeguards there must already be satisfied in the YAML, not left to the cluster. +**Provenance**: every generated K8s object carries `app.kubernetes.io/managed-by: aks-kickstart` on its own `metadata.labels` (not the pod template, not any selector); Dockerfiles carry the `com.azure.aks.kickstart.*` LABELs; Bicep resources carry the `managed-by` tag. A version marker is present and correct, or absent — never guessed. + **Bicep**: API versions pinned, parameterized env values, secure defaults (TLS 1.2+), outputs defined. **GitHub Actions**: OIDC auth, minimal `permissions`, environment protection for prod. diff --git a/skills/kickstart-safeguard-checklist/SKILL.md b/skills/kickstart-safeguard-checklist/SKILL.md index 4b00dd0d4..ccb5a3b91 100644 --- a/skills/kickstart-safeguard-checklist/SKILL.md +++ b/skills/kickstart-safeguard-checklist/SKILL.md @@ -21,7 +21,7 @@ It has **two parts**, and they are not interchangeable: |---|---|---|---| | A1 | Cannot Edit Individual Nodes | HIGH | N/A — rejected, not mutated | | A2 | Containers CPU and memory resource **requests** must be defined | HIGH | **Mutates** — sets default CPU/memory requests and enforces minimums | -| A3 | Must have anti-affinity rules or `topologySpreadConstraints` set | MEDIUM | **Mutates** — adds pod anti-affinity + topology spread constraints | +| A3 | Must have anti-affinity rules or `topologySpreadConstraints` set | MEDIUM | **Mutates** — adds pod anti-affinity + topology spread constraints (multi-replica workloads) | | A4 | No AKS-specific labels | MEDIUM | N/A | | A5 | Containers should only use allowed images | HIGH | N/A | | A6 | Reserved system pool taints | MEDIUM | **Mutates** — removes the `CriticalAddonsOnly` taint/toleration from user node pool workloads | @@ -36,16 +36,20 @@ It has **two parts**, and they are not interchangeable: ### A2: require-requests *(mutating)* - **Check**: Every container (including init containers and sidecars) declares `resources.requests.cpu` **and** `resources.requests.memory`. Limits alone are not sufficient. -- **If omitted**: AKS injects defaults and raises anything below the enforced minimum — your applied object will differ from your YAML. +- **If omitted**: the injected defaults are large — **CPU `500m` and memory `2048Mi` (2Gi) per container**, set as both request and limit. On a multi-container pod that is a real scheduling and cost surprise, so always declare requests explicitly. +- **Enforced minimums**: CPU `100m`, memory `100Mi`. Values below these are raised. If a request ends up above its limit the request is capped to the limit to keep the QoS class valid. - [ ] Pass / Fail ### A3: require-spread-or-anti-affinity *(mutating)* +- **Applies to**: multi-replica workloads. The policy error reads `Deployment with 2 replicas should have either podAntiAffinity or topologySpreadConstraints set` — single-replica workloads aren't flagged. Generate the constraints anyway, so scaling up later doesn't silently trigger a mutation. - **Check**: Each Deployment/StatefulSet sets `spec.template.spec.topologySpreadConstraints` **or** `affinity.podAntiAffinity`. Prefer topology spread on `kubernetes.io/hostname` with `whenUnsatisfiable: ScheduleAnyway`. -- **If omitted**: AKS adds its own anti-affinity and spread constraints, which can change scheduling behavior you didn't plan for. +- **If omitted**: AKS adds a preferred pod anti-affinity rule (weight 100, topology key `kubernetes.io/hostname`) plus a topology spread constraint (`maxSkew: 1`, `whenUnsatisfiable: ScheduleAnyway`). It picks the selector label by priority — `app`, then `app.kubernetes.io/name`, else a generated `default-antiaffinity-applabel=`. The mutator skips a workload entirely if *either* pod anti-affinity or any topology spread constraint already exists. - [ ] Pass / Fail ### A4: no-aks-specific-labels -- **Check**: No object sets a `kubernetes.azure.com/*` label. These are reserved for AKS. (`azure.workload.identity/*` labels and annotations are **not** covered by this rule and are required for Workload Identity.) +- **Check**: No object sets a `kubernetes.azure.com/*` **label**. These are reserved for AKS (`Label kubernetes.azure.com is reserved for AKS use only`). +- **Scope**: labels only. The policy does not inspect `metadata.annotations` — the safeguards docs name the annotation field explicitly where they mean it (e.g. the AppArmor rule). Even so, don't put `kubernetes.azure.com/*` in an annotation; the prefix is reserved by convention regardless of which field is enforced. +- `azure.workload.identity/*` labels and annotations are **not** covered by this rule and are required for Workload Identity. - [ ] Pass / Fail ### A5: allowed-images-only @@ -62,7 +66,8 @@ It has **two parts**, and they are not interchangeable: - [ ] Pass / Fail ### A8: csi-storageclass -- **Check**: Every PersistentVolumeClaim sets `storageClassName` to a CSI-backed class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`). No in-tree provisioners, no reliance on an unset default. +- **Check**: the policy evaluates the StorageClass **provisioner**, not its name. In-tree `kubernetes.io/azure-disk` and `kubernetes.io/azure-file` are rejected (`Storage class use intree provisioner ... is not allowed`); use `disk.csi.azure.com` or `file.csi.azure.com`. +- In practice: set `storageClassName` on every PVC to a CSI-backed class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`) rather than relying on an unset default, and verify the class's provisioner if it's cluster-custom. - [ ] Pass / Fail (N/A when no PVCs) ### A9: unique-service-selectors @@ -75,6 +80,13 @@ It has **two parts**, and they are not interchangeable: --- +### Enforcement caveats + +- **Gatekeeper runs fail-open.** If the admission webhook doesn't respond, validation is skipped and a non-compliant workload is admitted. Safeguards are a backstop, not a guarantee — generate compliant manifests rather than relying on enforcement to catch mistakes. +- **All or nothing.** Safeguards can't be enabled selectively; turning on `Warn` or `Enforce` activates every policy. Namespaces can be excluded, but on AKS Automatic the level can't be lowered from `Enforce`. + +--- + # Part B — Pod security & best practice ### Rule: no-privileged From c150a227859adc87b9b4920f6e56f344ae6b7725 Mon Sep 17 00:00:00 2001 From: David Gamero Date: Thu, 20 Aug 2026 17:18:10 -0400 Subject: [PATCH 3/4] docs(kickstart): pin the provenance version marker to v1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Uses the literal string v1 rather than the extension version. The marker now versions the kickstart artifact contract — the label/annotation scheme and the shape of what kickstart emits — instead of tracking releases, so it only changes when that contract changes. This also removes the dependency on injecting the extension version into agent context, which has no path today: the skills are static markdown with no interpolation. The previous "omit the version if you can't determine it" fallback is gone, so generated resources are now consistently stamped. Keeps the version as an annotation rather than a label. It is read, never selected on, and label values reject '+', so it could not hold a semver with build metadata if this later tracks a real release. --- skills/kickstart-generate/SKILL.md | 10 +++++----- skills/kickstart-review/SKILL.md | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/skills/kickstart-generate/SKILL.md b/skills/kickstart-generate/SKILL.md index 18faa9228..99bd9282e 100644 --- a/skills/kickstart-generate/SKILL.md +++ b/skills/kickstart-generate/SKILL.md @@ -40,21 +40,21 @@ metadata: labels: app.kubernetes.io/managed-by: aks-kickstart annotations: - kickstart.aks.azure.com/version: "" + kickstart.aks.azure.com/version: "v1" ``` Rules for placement, which matter more than the choice of field: - Put both on the **object's own `metadata`** (Deployment, Service, HTTPRoute, ServiceAccount, ConfigMap...), **not** on `spec.template.metadata`. Anything on the pod template — label or annotation — changes the pod-template hash and forces a full rollout on the next version bump. (`kubectl rollout restart` works precisely by writing a pod-template annotation.) - Never put either in `spec.selector` or in a Service selector. `spec.selector` is immutable after creation, and a version in a selector would make the Deployment un-updatable. It also collides with the unique-selector safeguard (A9). -- `app.kubernetes.io/managed-by` is the convention already used elsewhere in this extension, and its neutral prefix keeps it clear of the reserved `kubernetes.azure.com/*` namespace blocked by safeguard A4. The version is an **annotation** rather than a label because label values are capped at 63 chars and reject `+`, so a semver with build metadata (`2.4.0+abc123`) would be an invalid label value but is a perfectly fine annotation. +- `app.kubernetes.io/managed-by` is the convention already used elsewhere in this extension, and its neutral prefix keeps it clear of the reserved `kubernetes.azure.com/*` namespace blocked by safeguard A4. The version stays an **annotation** rather than a label: it's read, never selected on, and label values reject `+`, so it can't hold a semver with build metadata if this later tracks a real release. **Dockerfile** — OCI labels in the final stage. Use a kickstart-specific key for the generator version; `org.opencontainers.image.version` means the *application's* version, not the tool's: ```dockerfile LABEL org.opencontainers.image.source="" \ com.azure.aks.kickstart.generated="true" \ - com.azure.aks.kickstart.version="" + com.azure.aks.kickstart.version="v1" ``` **Bicep** — Azure resources take `tags`: @@ -62,11 +62,11 @@ LABEL org.opencontainers.image.source="" \ ```bicep tags: { 'managed-by': 'aks-kickstart' - 'kickstart-version': '' + 'kickstart-version': 'v1' } ``` -If you can't determine the extension version, emit the `managed-by` label / `generated` LABEL / `managed-by` tag and **omit the version entirely** — a wrong version is worse than an absent one, since it misattributes which release produced the artifact. Never guess or hardcode a version number. +**The version is the literal string `v1`** — it versions the kickstart artifact *contract* (this label/annotation scheme and the shape of what kickstart emits), not the extension release. Write `v1` verbatim; do not substitute the extension version, a semver, or a date. It changes only when the generated-artifact contract changes in a way consumers need to detect. ### Generate safeguard-compliant manifests up front diff --git a/skills/kickstart-review/SKILL.md b/skills/kickstart-review/SKILL.md index 17874a433..b69a32ff6 100644 --- a/skills/kickstart-review/SKILL.md +++ b/skills/kickstart-review/SKILL.md @@ -16,7 +16,7 @@ Validate every artifact against security, correctness, and AKS Automatic complia **K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, `topologySpreadConstraints` or anti-affinity, unique per-Service selectors, CSI `storageClassName` on any PVC, no `kubernetes.azure.com/*` labels, no `CriticalAddonsOnly` toleration, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. Run the full `/kickstart-safeguard-checklist` — the mutating safeguards there must already be satisfied in the YAML, not left to the cluster. -**Provenance**: every generated K8s object carries `app.kubernetes.io/managed-by: aks-kickstart` on its own `metadata.labels` (not the pod template, not any selector); Dockerfiles carry the `com.azure.aks.kickstart.*` LABELs; Bicep resources carry the `managed-by` tag. A version marker is present and correct, or absent — never guessed. +**Provenance**: every generated K8s object carries `app.kubernetes.io/managed-by: aks-kickstart` on its own `metadata.labels` (not the pod template, not any selector); Dockerfiles carry the `com.azure.aks.kickstart.*` LABELs; Bicep resources carry the `managed-by` tag. The version marker is the literal `v1` on all three. **Bicep**: API versions pinned, parameterized env values, secure defaults (TLS 1.2+), outputs defined. From a809fc8e79ae8df428f9cf1be510ef7cbeb613f8 Mon Sep 17 00:00:00 2001 From: David Gamero Date: Wed, 26 Aug 2026 18:21:06 +0000 Subject: [PATCH 4/4] fix(kickstart): address ACR build review feedback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1b48d1e8-85f6-4575-b25e-4b022d62eea7 --- AGENTS.md | 4 ++-- agents/kickstart-reviewer.agent.md | 4 ++-- agents/kickstart.agent.md | 6 +++--- kickstart-guide.md | 4 ++-- skills/kickstart-deploy/SKILL.md | 6 +++--- skills/kickstart-discover/SKILL.md | 8 ++------ skills/kickstart-generate/SKILL.md | 19 ++++++++----------- skills/kickstart-review/SKILL.md | 4 ++-- skills/kickstart-safeguard-checklist/SKILL.md | 15 +++++++++++---- 9 files changed, 35 insertions(+), 35 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4263f8e83..ac952f8e0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,8 +14,8 @@ Guides users through deploying an application to AKS Automatic in seven sequenti 1. **Discover** — Understand the app (language, deps, ports, env vars) and map each service's structure (build context, entry point, existing Dockerfile path) 2. **Configure Infrastructure** — Create new or select existing Azure resources (RG, AKS, ACR). Cluster creates with `--no-wait` to run in background. 3. **Design** — Propose target architecture, get approval -4. **Generate** — Create Dockerfile (reuse existing when present, validate every COPY/ADD path, build & inspect the image), K8s manifests, Bicep, GitHub Actions workflow -5. **Review** — Validate artifacts against safeguards and security checks, confirm the Dockerfile source→destination map, verify the image builds +4. **Generate** — Create Dockerfile (reuse existing when present, validate every COPY/ADD path, build remotely with ACR), K8s manifests, Bicep, GitHub Actions workflow +5. **Review** — Validate artifacts against safeguards and security checks, confirm the Dockerfile source→destination map resolves the entry point, verify the ACR build succeeds 6. **Pre-Deploy Check** — Verify cluster is ready, ACR attached 7. **Deploy** — Build (per-service build context + Dockerfile path, never `.`), push, apply with `az` CLI and `kubectl`, then health-check the running app diff --git a/agents/kickstart-reviewer.agent.md b/agents/kickstart-reviewer.agent.md index 10414f3b9..303808342 100644 --- a/agents/kickstart-reviewer.agent.md +++ b/agents/kickstart-reviewer.agent.md @@ -19,7 +19,7 @@ You review deployment artifacts generated by the Kickstart agent. Your job is to ## Review Process 1. Invoke `/kickstart-review` for the full per-artifact checklist (Dockerfile, Kubernetes manifests, Bicep, GitHub Actions) plus dry-run validation (`kubectl apply --dry-run=client`, `az bicep build`). -2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. Report **Part A** (policy-enforced AKS Deployment Safeguards) separately from **Part B** (pod security / best practice), and treat a FAIL on a *mutating* safeguard (A2 requests, A3 spread/anti-affinity, A6 reserved taints) as a real defect — the cluster will rewrite the object rather than reject it, so it must be fixed in the YAML. +2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. Report **Part A** (policy-enforced AKS Deployment Safeguards) separately from **Part B** (pod security / best practice), and treat a FAIL on a *mutating* safeguard (A2 requests, A3 spread/anti-affinity, and A6 when generated node-pool configuration is in scope) as a real defect — the cluster will rewrite the object rather than reject it, so it must be fixed in the generated artifact. 3. Invoke `/kickstart-security-hardening` for security-specific checks (RBAC scope, Key Vault, encryption, network isolation). Work through every generated artifact against those checklists — do not re-derive the criteria here. @@ -28,4 +28,4 @@ Work through every generated artifact against those checklists — do not re-der Present findings as a checklist with **PASS** ✓, **FAIL** ✗, or **WARN** ⚠ for each item. If any FAIL items exist, list specific fixes needed. -Images are always built with `az acr build`, never `docker build` — never ask for local Docker output or tell the user to install Docker. Verify the entry point via the Dockerfile's build-time `RUN test -f ` assertion. +Prefer `az acr build` for images; never require local Docker output or tell the user to install Docker. Verify the entry point by reconciling final-stage `COPY`/`ADD` destinations, `WORKDIR`, and `CMD`/`ENTRYPOINT`; do not inject shell commands into distroless or `scratch` final stages. diff --git a/agents/kickstart.agent.md b/agents/kickstart.agent.md index 211ca37b8..cc246f44a 100644 --- a/agents/kickstart.agent.md +++ b/agents/kickstart.agent.md @@ -85,15 +85,15 @@ Follow `/kickstart-configure-infra`. Do NOT pick subscriptions or run `az aks cr Follow `/kickstart-design`. Present architecture summary (container strategy, AKS Automatic, Gateway API, Workload Identity, ACR, monitoring). Get user approval via `vscode_askQuestions`. Run `/kickstart-cluster-status` before transitioning. ### 4 — Generate -Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Emit manifests that already satisfy the mutating AKS Deployment Safeguards (CPU/memory **requests**, `topologySpreadConstraints` or pod anti-affinity, unique per-service Service selectors, probes, CSI `storageClassName`, no `kubernetes.azure.com/*` labels or `CriticalAddonsOnly` toleration). Build each image with `az acr build` (server-side; never `docker build`) and assert the entry point with a `RUN test -f` line in the Dockerfile before exiting. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Emit manifests that already satisfy the mutating AKS Deployment Safeguards (CPU/memory **requests**, `topologySpreadConstraints` or pod anti-affinity, unique per-service Service selectors, probes, CSI `storageClassName`, no `kubernetes.azure.com/*` labels) and omit `CriticalAddonsOnly` tolerations from app workloads as a best practice. Preflight ACR build permissions, then build each image with `az acr build` and validate the final-stage source→destination map resolves the entry point. Run `/kickstart-cluster-status` before transitioning. ### 5 — Review -Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds with the entry point present via `az acr build` plus its build-time entry-point assertion — then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds via `az acr build` with its final-stage paths resolving the entry point — then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. ### 6 — Pre-Deploy Check Follow `/kickstart-handoff` — it carries the full strict-order playbook: cluster readiness (6a), metadata detection (6b), ACR attachment verification (6c — idempotent; the registry is usually already attached during cluster setup), kubelogin (6d), and the consolidated permission probes (6e–6g) via the bundled `aks.checkDeploymentPermissions` command. Escalate through `/kickstart-pim-activation` whenever a role assignment returns 403. Confirm readiness with the user via `vscode_askQuestions` before deploying. ### 7 — Deploy -Follow `/kickstart-deploy` — build & push to ACR with `az acr build` (server-side, never `docker build`; using each service's build context and Dockerfile path from the structure map, never `.`), get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. +Follow `/kickstart-deploy` — prefer building and pushing with server-side `az acr build`, using each service's build context and Dockerfile path from the structure map (never `.`); get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. Once the app is running, offer to commit the generated artifacts (Dockerfile, `k8s/`, `infra/`, workflow). If the user wants to commit or open a PR, follow `/kickstart-github-pr-conventions` for branch naming, Conventional Commits, and PR structure. diff --git a/kickstart-guide.md b/kickstart-guide.md index f38bb4635..0f604ff95 100644 --- a/kickstart-guide.md +++ b/kickstart-guide.md @@ -74,8 +74,8 @@ The main agent follows seven phases in strict order. Each phase has a dedicated | 1. Discover | `/kickstart-discover` | Collect app name, language, framework, deps, port, env vars, Dockerfile/CI status; map each service's build context, entry point, and existing Dockerfile path | Enough info to propose architecture + structure mapped | | 2. Configure | `/kickstart-configure-infra` | Create new or select existing Azure resources (RG, AKS cluster, ACR). Cluster creates with `--no-wait` | Resources selected/creating | | 3. Design | `/kickstart-design` | Propose AKS Automatic architecture, get user approval | User approves | -| 4. Generate | `/kickstart-generate` | Create Dockerfile (reuse existing, validate COPY/ADD paths, build & inspect image), K8s manifests, Bicep, GHA workflow | All files written + image builds | -| 5. Review | `/kickstart-review` | Confirm Dockerfile source→destination map, verify image builds, validate artifacts against safeguards + security | All checks pass | +| 4. Generate | `/kickstart-generate` | Create Dockerfile (reuse existing, validate COPY/ADD paths, build remotely with ACR), K8s manifests, Bicep, GHA workflow | All files written + ACR build succeeds | +| 5. Review | `/kickstart-review` | Confirm Dockerfile source→destination map resolves the entry point, verify ACR build, validate artifacts against safeguards + security | All checks pass | | 6. Pre-Deploy | `/kickstart-handoff` | Verify cluster ready, ACR attached, final summary | Cluster provisioned, user confirms | | 7. Deploy | `/kickstart-deploy` | Build (per-service context + Dockerfile path), push, apply with `az` and `kubectl`, health-check endpoint | App running + endpoint verified | diff --git a/skills/kickstart-deploy/SKILL.md b/skills/kickstart-deploy/SKILL.md index fdc6c5f77..4bd426cc1 100644 --- a/skills/kickstart-deploy/SKILL.md +++ b/skills/kickstart-deploy/SKILL.md @@ -11,16 +11,16 @@ Deploy using Azure CLI and `kubectl`. Execute each step via `run_in_terminal`, c ## Steps 1. **Build and push**: `az acr build --registry --image : -f ` - `az acr build` runs **server-side on the ACR remote task builders** — always use it, never `docker build` + `docker push`. Kickstart runs in Azure Cloud Shell, which has no Docker daemon, and a single remote build keeps the pushed digest and the deployed image identical. + Prefer `az acr build` over `docker build` + `docker push`. It runs on ACR's remote task builders, avoids a local Docker daemon dependency, and keeps the pushed digest and deployed image on one build path. Use the build context and Dockerfile path from the structure map — never assume repo root (`.`). For monorepos, build each service from its own context. Tag with a version (e.g. v1.0.0), never `:latest`. - Requires `AcrPush` or `Container Registry Tasks Contributor` (verified in Pre-Deploy Check). Keep the command in the foreground so its streamed log holds an idle Cloud Shell session open, and make sure `.dockerignore` excludes bulk (the whole context is uploaded on each build). + Requires the ACR build and push permissions verified in Generate and Pre-Deploy. Make sure `.dockerignore` excludes bulk because the whole context is uploaded on each build. Third-party images the app depends on (Redis, Postgres, RabbitMQ) can't be `az acr build`-ed — bring them in with `az acr import --name --source docker.io/library/: --image :`. 2. **Get credentials**: `az aks get-credentials --resource-group --name --overwrite-existing` kubelogin handles AAD auth automatically (verified in Pre-Deploy Check). Never use `--admin`. 3. **Apply manifests**: preview with `kubectl apply --dry-run=server -f k8s/`, then `kubectl apply -f k8s/`. - Expect mutation: AKS Automatic's Deployment Safeguards rewrite on admission — missing CPU/memory requests get defaults, missing spread rules get anti-affinity/topology constraints, a stray `CriticalAddonsOnly` toleration is stripped. If Phase 4 generated compliant manifests these are no-ops; if `kubectl get -o yaml` differs from your file, that's the mutator, not a bug. + Expect mutation: AKS Automatic's Deployment Safeguards rewrite on admission — missing CPU/memory requests get defaults and missing spread rules get anti-affinity/topology constraints. If Phase 4 generated compliant manifests these are no-ops; if `kubectl get -o yaml` differs from your file, inspect the safeguard mutation before treating it as a bug. 4. **Verify**: `kubectl get pods -n ` and `kubectl get services -n ` If pods not Ready, run `kubectl describe pod ` and `kubectl logs ` to diagnose. diff --git a/skills/kickstart-discover/SKILL.md b/skills/kickstart-discover/SKILL.md index 16669a98e..c3016f028 100644 --- a/skills/kickstart-discover/SKILL.md +++ b/skills/kickstart-discover/SKILL.md @@ -28,13 +28,9 @@ Use `codebase`/`search` to confirm each path actually exists — do not infer it ## Build environment -All container images are built with **`az acr build`** — server-side on the ACR remote task builders. Never `docker build`. Kickstart runs in Azure Cloud Shell, which has no Docker daemon, and a single remote build path keeps behavior identical everywhere. +Prefer **`az acr build`** for container images. It builds on ACR's remote task builders, avoids depending on a local Docker daemon, and uses the same build-and-push path in every environment. -Cloud Shell constraints to honor: - -- Clone into `~/clouddrive/` (persistent, ~5 GB) rather than the ephemeral home dir. -- `az acr build` needs the Phase 2 ACR to exist and the caller to hold `AcrPush` (or `Container Registry Tasks Contributor`) — see `/kickstart-handoff`. -- Keep `az acr build` in the foreground so its streamed log holds an idle session open, and keep `.dockerignore` tight (the whole build context uploads on every build). +`az acr build` needs the Phase 2 ACR to exist and the caller to hold the required ACR build and push permissions. Keep `.dockerignore` tight because the whole build context uploads on every build. ## What to collect diff --git a/skills/kickstart-generate/SKILL.md b/skills/kickstart-generate/SKILL.md index 99bd9282e..1afaf683d 100644 --- a/skills/kickstart-generate/SKILL.md +++ b/skills/kickstart-generate/SKILL.md @@ -90,7 +90,7 @@ AKS Automatic enforces **Deployment Safeguards**, and several of them *mutate* y - **Readiness *and* liveness probes** on every container — use the real health path and port from the structure map, never a guessed `/healthz`. - **Unique Service selectors** — each Service must select exactly one workload. In a monorepo do **not** reuse `app: ` across services; use `app: ` (or `app.kubernetes.io/name` + `app.kubernetes.io/component`) so no two Services overlap. - **No AKS-specific labels** — never set `kubernetes.azure.com/*` labels on your own objects. (`azure.workload.identity/*` labels and annotations are fine and required.) -- **No reserved system-pool taints/tolerations** — do not add a `CriticalAddonsOnly` toleration to app pods; it would place them on the system pool and AKS strips it anyway. +- **No reserved system-pool taints** — never configure `CriticalAddonsOnly` on a user node pool. Also do not add a matching toleration to app pods: a toleration permits, but does not force, scheduling onto a matching tainted node and is inappropriate for ordinary workloads. - **CSI StorageClass for any PVC** — set `storageClassName` explicitly to a CSI class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`); never rely on an in-tree or unset default. - **Pinned image tags** — no `:latest` anywhere, including init containers and sidecars. - **Allowed images only** — every image must come from the Phase 2 ACR (`.azurecr.io/...`) if the cluster restricts registries; flag any third-party image (Redis, Postgres, RabbitMQ) that would need importing via `az acr import`. @@ -106,25 +106,22 @@ AKS Automatic enforces **Deployment Safeguards**, and several of them *mutate* y - Honor each service's build context and entry point from the structure map; reuse existing Dockerfiles instead of duplicating them. - Stamp every generated resource with the kickstart provenance marker above (object metadata only — never the pod template or a selector). - All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath) **and** must pre-satisfy the mutating safeguards above so the cluster doesn't rewrite them on apply. -- Build images with `az acr build` only — never `docker build`. +- Prefer `az acr build` for image builds instead of `docker build`. - After writing all files, confirm with user via `vscode_askQuestions`. ## Validate the build (before exit) -Do not hand off unbuilt artifacts. Build every Dockerfile with **`az acr build`** — server-side on the ACR remote task builders. Never `docker build`; there is no Docker daemon in Cloud Shell, and one build path keeps the validated image and the deployed image identical. +Do not hand off unbuilt artifacts. Build every Dockerfile with **`az acr build`** on the ACR remote task builders. Prefer this over `docker build` so validation does not depend on a local daemon and uses the same build-and-push path as deployment. -1. **Build in ACR** from the service's own build context. This also catches missing `COPY`/`ADD` sources: +1. **Preflight ACR permissions before the first build.** Invoke `aks.checkDeploymentPermissions` via `vscode/runCommand` with the Phase 2 resource names, `probeScope: "user"`, and `silent: true`. Inspect the `acr-push` and `acr-tasks` probes. If either fails, offer its included remediation; if role assignment returns 403, follow `/kickstart-pim-activation`. Do not wait until Pre-Deploy to surface an ACR permission failure. +2. **Build in ACR** from the service's own build context. This catches missing `COPY`/`ADD` sources: ```bash az acr build --registry --image kickstart-validate/:check -f ``` -2. **Assert the entry point at build time.** Add to the Dockerfile's final stage so a wrong path fails the ACR build itself: - ```dockerfile - RUN test -f - ``` - The assertion runs before the cluster exists, so a wrong path is caught immediately. -3. If the build fails or the assertion trips, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. +3. **Verify the final image layout from the Dockerfile.** Reconcile every final-stage `COPY`/`ADD` destination with `WORKDIR` and `CMD`/`ENTRYPOINT`, then present that source→destination map in Review. Do not inject `RUN test -f` into the final stage: distroless and `scratch` images have no shell, and a path-existence check does not prove the entry point landed in the intended directory. +4. If the build fails or the source→destination map does not resolve the entry point, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. Keep `.dockerignore` tight: the entire build context is uploaded to ACR on every build. ## Exit Criteria -All artifacts written, every Dockerfile builds via `az acr build`, and the build-time entry-point assertion passes. Announce: "Artifacts generated and build-validated — moving to Review." +All artifacts written, ACR build permissions confirmed, every Dockerfile builds via `az acr build`, and the final-stage source→destination map resolves the entry point. Announce: "Artifacts generated and build-validated — moving to Review." diff --git a/skills/kickstart-review/SKILL.md b/skills/kickstart-review/SKILL.md index b69a32ff6..ed79c3c92 100644 --- a/skills/kickstart-review/SKILL.md +++ b/skills/kickstart-review/SKILL.md @@ -12,7 +12,7 @@ Validate every artifact against security, correctness, and AKS Automatic complia **Dockerfile**: Multi-stage build, pinned base image, non-root user, `.dockerignore` present. Build context + every `COPY`/`ADD` source→destination resolves to real files; `CMD`/`ENTRYPOINT` runs the actual entry point; the image builds and the entry point is present in the built image. -**Dockerfile build**: built with `az acr build` (never `docker build`), with a `RUN test -f ` assertion in the final stage. +**Dockerfile build**: built with `az acr build`; final-stage `COPY`/`ADD` destinations, `WORKDIR`, and `CMD`/`ENTRYPOINT` collectively resolve the real entry point. Do not require a shell-based assertion in distroless or `scratch` images. **K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, `topologySpreadConstraints` or anti-affinity, unique per-Service selectors, CSI `storageClassName` on any PVC, no `kubernetes.azure.com/*` labels, no `CriticalAddonsOnly` toleration, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. Run the full `/kickstart-safeguard-checklist` — the mutating safeguards there must already be satisfied in the YAML, not left to the cluster. @@ -30,7 +30,7 @@ Validate every artifact against security, correctness, and AKS Automatic complia |---|---|---|---|---| | `src/order-service` | `package.json`, `src/` | `/app` | `/app/server.js` | 3000 | - Then confirm (or run) the build validation from `/kickstart-generate`: the image must build via `az acr build`, and the Dockerfile's build-time `RUN test -f ` assertion must be present and passing. A missing assertion or a failed build is a FAIL. Do not ask for `docker build` / `docker run` output — images are always built server-side in ACR. + Then confirm (or run) the build validation from `/kickstart-generate`: the image must build via `az acr build`, and the final-stage source→destination map must resolve the declared entry point. A failed build or unresolved entry point is a FAIL. Do not require local Docker output; prefer the server-side ACR build. 2. Run `/kickstart-safeguard-checklist` for the full safeguard rule set. 3. Run validation via `run_in_terminal`: diff --git a/skills/kickstart-safeguard-checklist/SKILL.md b/skills/kickstart-safeguard-checklist/SKILL.md index ccb5a3b91..ac708f13d 100644 --- a/skills/kickstart-safeguard-checklist/SKILL.md +++ b/skills/kickstart-safeguard-checklist/SKILL.md @@ -24,7 +24,7 @@ It has **two parts**, and they are not interchangeable: | A3 | Must have anti-affinity rules or `topologySpreadConstraints` set | MEDIUM | **Mutates** — adds pod anti-affinity + topology spread constraints (multi-replica workloads) | | A4 | No AKS-specific labels | MEDIUM | N/A | | A5 | Containers should only use allowed images | HIGH | N/A | -| A6 | Reserved system pool taints | MEDIUM | **Mutates** — removes the `CriticalAddonsOnly` taint/toleration from user node pool workloads | +| A6 | Reserved system pool taints | MEDIUM | **Mutates** — removes the `CriticalAddonsOnly` taint from user node pools | | A7 | Containers have readiness or liveness probes configured | HIGH | N/A | | A8 | Clusters should use CSI driver StorageClass | MEDIUM | N/A | | A9 | Services should use unique selectors | HIGH | N/A | @@ -57,9 +57,10 @@ It has **two parts**, and they are not interchangeable: - [ ] Pass / Fail ### A6: reserved-system-pool-taints *(mutating)* -- **Check**: No app workload declares a `CriticalAddonsOnly` toleration, and no user node pool config sets that taint. AKS uses it to keep customer pods off the system pool. -- **If present**: AKS removes it, so any scheduling you based on it will not hold. -- [ ] Pass / Fail +- **Applies to**: Node and node-pool configuration, not Pod tolerations. The policy evaluates node `spec.taints` and excludes system pools by their `kubernetes.azure.com/mode` label. +- **Check**: No generated user node pool config sets the `CriticalAddonsOnly` taint. AKS reserves it for system pools. +- **If present**: AKS removes the taint from the user pool. +- [ ] Pass / Fail (N/A when no user node pool configuration is generated) ### A7: require-probes - **Check**: Every container defines a `readinessProbe` **or** `livenessProbe` (Kickstart generates both). The path and port must match the app's real health endpoint from the structure map — not a guessed `/healthz`. @@ -89,6 +90,12 @@ It has **two parts**, and they are not interchangeable: # Part B — Pod security & best practice +### Rule: no-system-pool-toleration +- **Severity**: MEDIUM +- **Description**: Ordinary app pods should not tolerate the system pool's `CriticalAddonsOnly` taint. A toleration permits scheduling onto a matching tainted node but does not attract the pod there; this is a Kickstart best practice, not the A6 policy check. +- **Check**: Verify app workloads do not declare a `CriticalAddonsOnly` toleration. +- [ ] Pass / Fail + ### Rule: no-privileged - **Severity**: HIGH - **Description**: Containers must not run in privileged mode.