diff --git a/AGENTS.md b/AGENTS.md index 4263f8e83..ac952f8e0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,8 +14,8 @@ Guides users through deploying an application to AKS Automatic in seven sequenti 1. **Discover** — Understand the app (language, deps, ports, env vars) and map each service's structure (build context, entry point, existing Dockerfile path) 2. **Configure Infrastructure** — Create new or select existing Azure resources (RG, AKS, ACR). Cluster creates with `--no-wait` to run in background. 3. **Design** — Propose target architecture, get approval -4. **Generate** — Create Dockerfile (reuse existing when present, validate every COPY/ADD path, build & inspect the image), K8s manifests, Bicep, GitHub Actions workflow -5. **Review** — Validate artifacts against safeguards and security checks, confirm the Dockerfile source→destination map, verify the image builds +4. **Generate** — Create Dockerfile (reuse existing when present, validate every COPY/ADD path, build remotely with ACR), K8s manifests, Bicep, GitHub Actions workflow +5. **Review** — Validate artifacts against safeguards and security checks, confirm the Dockerfile source→destination map resolves the entry point, verify the ACR build succeeds 6. **Pre-Deploy Check** — Verify cluster is ready, ACR attached 7. **Deploy** — Build (per-service build context + Dockerfile path, never `.`), push, apply with `az` CLI and `kubectl`, then health-check the running app diff --git a/agents/kickstart-reviewer.agent.md b/agents/kickstart-reviewer.agent.md index 0b2fdefaf..303808342 100644 --- a/agents/kickstart-reviewer.agent.md +++ b/agents/kickstart-reviewer.agent.md @@ -19,7 +19,7 @@ You review deployment artifacts generated by the Kickstart agent. Your job is to ## Review Process 1. Invoke `/kickstart-review` for the full per-artifact checklist (Dockerfile, Kubernetes manifests, Bicep, GitHub Actions) plus dry-run validation (`kubectl apply --dry-run=client`, `az bicep build`). -2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. +2. Invoke `/kickstart-safeguard-checklist` to validate manifests against AKS deployment safeguard rules. Report **Part A** (policy-enforced AKS Deployment Safeguards) separately from **Part B** (pod security / best practice), and treat a FAIL on a *mutating* safeguard (A2 requests, A3 spread/anti-affinity, and A6 when generated node-pool configuration is in scope) as a real defect — the cluster will rewrite the object rather than reject it, so it must be fixed in the generated artifact. 3. Invoke `/kickstart-security-hardening` for security-specific checks (RBAC scope, Key Vault, encryption, network isolation). Work through every generated artifact against those checklists — do not re-derive the criteria here. @@ -27,3 +27,5 @@ Work through every generated artifact against those checklists — do not re-der ## Output Present findings as a checklist with **PASS** ✓, **FAIL** ✗, or **WARN** ⚠ for each item. If any FAIL items exist, list specific fixes needed. + +Prefer `az acr build` for images; never require local Docker output or tell the user to install Docker. Verify the entry point by reconciling final-stage `COPY`/`ADD` destinations, `WORKDIR`, and `CMD`/`ENTRYPOINT`; do not inject shell commands into distroless or `scratch` final stages. diff --git a/agents/kickstart.agent.md b/agents/kickstart.agent.md index 958bae773..cc246f44a 100644 --- a/agents/kickstart.agent.md +++ b/agents/kickstart.agent.md @@ -66,7 +66,7 @@ Handle each starting point (ask via `vscode_askQuestions` only when it wasn't al }] } ``` - Clone with `run_in_terminal`, then load `/kickstart-samples` for the pre-filled profile and confirm it with the user. **Skip Phase 1's questions** — go straight to **Phase 2 (Configure Infrastructure)**. Do NOT ask the user for app name, port, language, or any discovery questions — but still run the quick structure scan from `/kickstart-samples` to confirm each service's build context, Dockerfile path, and entry point before generating anything. + Clone with `run_in_terminal`, then load `/kickstart-samples` for the pre-filled profile and confirm it with the user. **Skip Phase 1's questions** — go straight to **Phase 2 (Configure Infrastructure)**. Do NOT ask the user for app name, port, language, or any discovery questions — but still run the quick structure scan from `/kickstart-samples` to confirm each service's build context, Dockerfile path, and entry point, before generating anything. - **Use my current workspace**: Proceed to **Phase 1 (Discover)**. ## Phases @@ -85,15 +85,15 @@ Follow `/kickstart-configure-infra`. Do NOT pick subscriptions or run `az aks cr Follow `/kickstart-design`. Present architecture summary (container strategy, AKS Automatic, Gateway API, Workload Identity, ACR, monitoring). Get user approval via `vscode_askQuestions`. Run `/kickstart-cluster-status` before transitioning. ### 4 — Generate -Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Build and inspect each image (confirm the entry point landed) before exiting. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-generate`. Produce Dockerfile (reuse an existing one when present), K8s manifests (`k8s/`), Bicep (`infra/`), GitHub Actions workflow — driven by the structure map, with every `COPY`/`ADD` path validated against the build context. Use actual resource names from Phase 2. Pin image tags — never `:latest`. Emit manifests that already satisfy the mutating AKS Deployment Safeguards (CPU/memory **requests**, `topologySpreadConstraints` or pod anti-affinity, unique per-service Service selectors, probes, CSI `storageClassName`, no `kubernetes.azure.com/*` labels) and omit `CriticalAddonsOnly` tolerations from app workloads as a best practice. Preflight ACR build permissions, then build each image with `az acr build` and validate the final-stage source→destination map resolves the entry point. Run `/kickstart-cluster-status` before transitioning. ### 5 — Review -Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds with the entry point present, then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. +Follow `/kickstart-review`. Show the Dockerfile source→destination map for confirmation and verify the image builds via `az acr build` with its final-stage paths resolving the entry point — then run `/kickstart-safeguard-checklist` validation. Present pass/fail/warn checklist. Fix failures before proceeding. Run `/kickstart-cluster-status` before transitioning. ### 6 — Pre-Deploy Check Follow `/kickstart-handoff` — it carries the full strict-order playbook: cluster readiness (6a), metadata detection (6b), ACR attachment verification (6c — idempotent; the registry is usually already attached during cluster setup), kubelogin (6d), and the consolidated permission probes (6e–6g) via the bundled `aks.checkDeploymentPermissions` command. Escalate through `/kickstart-pim-activation` whenever a role assignment returns 403. Confirm readiness with the user via `vscode_askQuestions` before deploying. ### 7 — Deploy -Follow `/kickstart-deploy` — build & push to ACR (using each service's build context and Dockerfile path from the structure map, never `.`), get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. +Follow `/kickstart-deploy` — prefer building and pushing with server-side `az acr build`, using each service's build context and Dockerfile path from the structure map (never `.`); get credentials, apply manifests, then verify and health-check the running app (hit its endpoint, compare expected vs actual — not just pod readiness), executed step by step via `run_in_terminal` with confirmation between each and error classification on failure. Once the app is running, offer to commit the generated artifacts (Dockerfile, `k8s/`, `infra/`, workflow). If the user wants to commit or open a PR, follow `/kickstart-github-pr-conventions` for branch naming, Conventional Commits, and PR structure. diff --git a/kickstart-guide.md b/kickstart-guide.md index f38bb4635..0f604ff95 100644 --- a/kickstart-guide.md +++ b/kickstart-guide.md @@ -74,8 +74,8 @@ The main agent follows seven phases in strict order. Each phase has a dedicated | 1. Discover | `/kickstart-discover` | Collect app name, language, framework, deps, port, env vars, Dockerfile/CI status; map each service's build context, entry point, and existing Dockerfile path | Enough info to propose architecture + structure mapped | | 2. Configure | `/kickstart-configure-infra` | Create new or select existing Azure resources (RG, AKS cluster, ACR). Cluster creates with `--no-wait` | Resources selected/creating | | 3. Design | `/kickstart-design` | Propose AKS Automatic architecture, get user approval | User approves | -| 4. Generate | `/kickstart-generate` | Create Dockerfile (reuse existing, validate COPY/ADD paths, build & inspect image), K8s manifests, Bicep, GHA workflow | All files written + image builds | -| 5. Review | `/kickstart-review` | Confirm Dockerfile source→destination map, verify image builds, validate artifacts against safeguards + security | All checks pass | +| 4. Generate | `/kickstart-generate` | Create Dockerfile (reuse existing, validate COPY/ADD paths, build remotely with ACR), K8s manifests, Bicep, GHA workflow | All files written + ACR build succeeds | +| 5. Review | `/kickstart-review` | Confirm Dockerfile source→destination map resolves the entry point, verify ACR build, validate artifacts against safeguards + security | All checks pass | | 6. Pre-Deploy | `/kickstart-handoff` | Verify cluster ready, ACR attached, final summary | Cluster provisioned, user confirms | | 7. Deploy | `/kickstart-deploy` | Build (per-service context + Dockerfile path), push, apply with `az` and `kubectl`, health-check endpoint | App running + endpoint verified | diff --git a/skills/kickstart-deploy/SKILL.md b/skills/kickstart-deploy/SKILL.md index 69405e518..4bd426cc1 100644 --- a/skills/kickstart-deploy/SKILL.md +++ b/skills/kickstart-deploy/SKILL.md @@ -11,12 +11,16 @@ Deploy using Azure CLI and `kubectl`. Execute each step via `run_in_terminal`, c ## Steps 1. **Build and push**: `az acr build --registry --image : -f ` + Prefer `az acr build` over `docker build` + `docker push`. It runs on ACR's remote task builders, avoids a local Docker daemon dependency, and keeps the pushed digest and deployed image on one build path. Use the build context and Dockerfile path from the structure map — never assume repo root (`.`). For monorepos, build each service from its own context. Tag with a version (e.g. v1.0.0), never `:latest`. + Requires the ACR build and push permissions verified in Generate and Pre-Deploy. Make sure `.dockerignore` excludes bulk because the whole context is uploaded on each build. + Third-party images the app depends on (Redis, Postgres, RabbitMQ) can't be `az acr build`-ed — bring them in with `az acr import --name --source docker.io/library/: --image :`. 2. **Get credentials**: `az aks get-credentials --resource-group --name --overwrite-existing` kubelogin handles AAD auth automatically (verified in Pre-Deploy Check). Never use `--admin`. -3. **Apply manifests**: `kubectl apply -f k8s/` +3. **Apply manifests**: preview with `kubectl apply --dry-run=server -f k8s/`, then `kubectl apply -f k8s/`. + Expect mutation: AKS Automatic's Deployment Safeguards rewrite on admission — missing CPU/memory requests get defaults and missing spread rules get anti-affinity/topology constraints. If Phase 4 generated compliant manifests these are no-ops; if `kubectl get -o yaml` differs from your file, inspect the safeguard mutation before treating it as a bug. 4. **Verify**: `kubectl get pods -n ` and `kubectl get services -n ` If pods not Ready, run `kubectl describe pod ` and `kubectl logs ` to diagnose. @@ -33,6 +37,7 @@ Classify failures: - **config** — missing or invalid configuration (subscription, RG, cluster, ACR, manifest) - **dependency** — missing CLI tool, extension, or container image - **cluster** — pod CrashLoopBackOff, ImagePullBackOff, scheduling, quota +- **safeguard** — admission rejected by a Deployment Safeguard (e.g. `:latest` tag, non-CSI StorageClass, duplicate Service selector, AKS-reserved label, node edit). Fix the manifest to satisfy the rule; never disable the policy. Provide specific `az` or `kubectl` fix commands. Offer retry via `vscode_askQuestions`. diff --git a/skills/kickstart-discover/SKILL.md b/skills/kickstart-discover/SKILL.md index 90bd0c36f..c3016f028 100644 --- a/skills/kickstart-discover/SKILL.md +++ b/skills/kickstart-discover/SKILL.md @@ -26,6 +26,12 @@ Never assume a flat repo. Apps often live in nested or monorepo layouts (`src/ } + ``` +- **Readiness *and* liveness probes** on every container — use the real health path and port from the structure map, never a guessed `/healthz`. +- **Unique Service selectors** — each Service must select exactly one workload. In a monorepo do **not** reuse `app: ` across services; use `app: ` (or `app.kubernetes.io/name` + `app.kubernetes.io/component`) so no two Services overlap. +- **No AKS-specific labels** — never set `kubernetes.azure.com/*` labels on your own objects. (`azure.workload.identity/*` labels and annotations are fine and required.) +- **No reserved system-pool taints** — never configure `CriticalAddonsOnly` on a user node pool. Also do not add a matching toleration to app pods: a toleration permits, but does not force, scheduling onto a matching tainted node and is inappropriate for ordinary workloads. +- **CSI StorageClass for any PVC** — set `storageClassName` explicitly to a CSI class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`); never rely on an in-tree or unset default. +- **Pinned image tags** — no `:latest` anywhere, including init containers and sidecars. +- **Allowed images only** — every image must come from the Phase 2 ACR (`.azurecr.io/...`) if the cluster restricts registries; flag any third-party image (Redis, Postgres, RabbitMQ) that would need importing via `az acr import`. +- **Never edit individual nodes** — no node-targeted manifests, `kubectl label node`, `kubectl taint node`, or node-name `nodeSelector`. Use node pools instead. **Bicep** (`infra/main.bicep`): AKS Automatic + ACR + Managed Identity + federated credential. Parameterized, pinned API versions. ARM resource IDs follow `/subscriptions/{sub}/resourceGroups/{rg}/providers/{ns}/{type}/{name}`. See `/kickstart-bicep-authoring` and `/kickstart-acr-integration`. @@ -37,18 +104,24 @@ Load these for detailed patterns as you author each artifact: - Use actual resource names from the Configure phase. - Never use `:latest` tags. - Honor each service's build context and entry point from the structure map; reuse existing Dockerfiles instead of duplicating them. -- All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath). +- Stamp every generated resource with the kickstart provenance marker above (object metadata only — never the pod template or a selector). +- All K8s manifests must comply with AKS deployment safeguards (restricted pod security, no privileged, no hostPath) **and** must pre-satisfy the mutating safeguards above so the cluster doesn't rewrite them on apply. +- Prefer `az acr build` for image builds instead of `docker build`. - After writing all files, confirm with user via `vscode_askQuestions`. ## Validate the build (before exit) -Do not hand off unbuilt artifacts. For each Dockerfile, build and inspect before announcing completion: +Do not hand off unbuilt artifacts. Build every Dockerfile with **`az acr build`** on the ACR remote task builders. Prefer this over `docker build` so validation does not depend on a local daemon and uses the same build-and-push path as deployment. -1. **Build** from the service's build context: - - Local Docker/Podman daemon available: `docker build -t kickstart-validate-:check -f `. - - Otherwise build in ACR (also catches missing `COPY` sources): `az acr build --registry --image kickstart-validate/:check -f `. -2. **Inspect contents** (when built locally): `docker run --rm kickstart-validate-:check ls -la ` — confirm the entry point and expected files landed where the app runs from. A build that succeeds but places files in the wrong dir is exactly the failure this step catches. -3. If the build fails or the entry point is missing, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. +1. **Preflight ACR permissions before the first build.** Invoke `aks.checkDeploymentPermissions` via `vscode/runCommand` with the Phase 2 resource names, `probeScope: "user"`, and `silent: true`. Inspect the `acr-push` and `acr-tasks` probes. If either fails, offer its included remediation; if role assignment returns 403, follow `/kickstart-pim-activation`. Do not wait until Pre-Deploy to surface an ACR permission failure. +2. **Build in ACR** from the service's own build context. This catches missing `COPY`/`ADD` sources: + ```bash + az acr build --registry --image kickstart-validate/:check -f + ``` +3. **Verify the final image layout from the Dockerfile.** Reconcile every final-stage `COPY`/`ADD` destination with `WORKDIR` and `CMD`/`ENTRYPOINT`, then present that source→destination map in Review. Do not inject `RUN test -f` into the final stage: distroless and `scratch` images have no shell, and a path-existence check does not prove the entry point landed in the intended directory. +4. If the build fails or the source→destination map does not resolve the entry point, fix the Dockerfile/paths and rebuild — do not proceed to Review with a broken image. + +Keep `.dockerignore` tight: the entire build context is uploaded to ACR on every build. ## Exit Criteria -All artifacts written, every Dockerfile builds, and the entry point is confirmed present in the image. Announce: "Artifacts generated and build-validated — moving to Review." +All artifacts written, ACR build permissions confirmed, every Dockerfile builds via `az acr build`, and the final-stage source→destination map resolves the entry point. Announce: "Artifacts generated and build-validated — moving to Review." diff --git a/skills/kickstart-review/SKILL.md b/skills/kickstart-review/SKILL.md index 9ddfffaf7..ed79c3c92 100644 --- a/skills/kickstart-review/SKILL.md +++ b/skills/kickstart-review/SKILL.md @@ -12,7 +12,11 @@ Validate every artifact against security, correctness, and AKS Automatic complia **Dockerfile**: Multi-stage build, pinned base image, non-root user, `.dockerignore` present. Build context + every `COPY`/`ADD` source→destination resolves to real files; `CMD`/`ENTRYPOINT` runs the actual entry point; the image builds and the entry point is present in the built image. -**K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. +**Dockerfile build**: built with `az acr build`; final-stage `COPY`/`ADD` destinations, `WORKDIR`, and `CMD`/`ENTRYPOINT` collectively resolve the real entry point. Do not require a shell-based assertion in distroless or `scratch` images. + +**K8s Manifests**: `runAsNonRoot: true`, no privileged containers, resource requests+limits, liveness/readiness probes, `topologySpreadConstraints` or anti-affinity, unique per-Service selectors, CSI `storageClassName` on any PVC, no `kubernetes.azure.com/*` labels, no `CriticalAddonsOnly` toleration, Gateway API HTTPRoute (not Ingress), Workload Identity labels+SA, namespace specified. Run the full `/kickstart-safeguard-checklist` — the mutating safeguards there must already be satisfied in the YAML, not left to the cluster. + +**Provenance**: every generated K8s object carries `app.kubernetes.io/managed-by: aks-kickstart` on its own `metadata.labels` (not the pod template, not any selector); Dockerfiles carry the `com.azure.aks.kickstart.*` LABELs; Bicep resources carry the `managed-by` tag. The version marker is the literal `v1` on all three. **Bicep**: API versions pinned, parameterized env values, secure defaults (TLS 1.2+), outputs defined. @@ -26,13 +30,15 @@ Validate every artifact against security, correctness, and AKS Automatic complia |---|---|---|---|---| | `src/order-service` | `package.json`, `src/` | `/app` | `/app/server.js` | 3000 | - Then confirm (or run) the build validation from `/kickstart-generate` — the image must build and `ls ` must show the entry point. A missing or mismatched path is a FAIL. + Then confirm (or run) the build validation from `/kickstart-generate`: the image must build via `az acr build`, and the final-stage source→destination map must resolve the declared entry point. A failed build or unresolved entry point is a FAIL. Do not require local Docker output; prefer the server-side ACR build. + 2. Run `/kickstart-safeguard-checklist` for the full safeguard rule set. 3. Run validation via `run_in_terminal`: ```bash kubectl apply --dry-run=client -f k8s/ az bicep build --file infra/main.bicep ``` + `--dry-run=server` is preferable when the cluster is reachable: it runs the admission webhooks, so it surfaces real Deployment Safeguard violations and shows the mutated result. 4. Present results as PASS ✓ / FAIL ✗ / WARN ⚠ per item. 5. If FAILs: use `vscode_askQuestions` — fix automatically (recommended), show details, or skip. 6. If WARNs only: confirm proceeding via `vscode_askQuestions`. diff --git a/skills/kickstart-safeguard-checklist/SKILL.md b/skills/kickstart-safeguard-checklist/SKILL.md index 40bb2b830..ac708f13d 100644 --- a/skills/kickstart-safeguard-checklist/SKILL.md +++ b/skills/kickstart-safeguard-checklist/SKILL.md @@ -6,9 +6,95 @@ disable-model-invocation: true # AKS Deployment Safeguard Checklist -This skill provides a comprehensive checklist for validating generated Kubernetes manifests against AKS security and deployment best practices. Use this during the Review phase to ensure all generated configurations comply with organizational policies. +This skill provides a comprehensive checklist for validating generated Kubernetes manifests. Use this during the Review phase. -## Safeguard Rules +It has **two parts**, and they are not interchangeable: + +- **Part A — AKS Deployment Safeguards.** The policies AKS Automatic actually enforces at admission. Some **mutate** your object instead of rejecting it, so a manifest that omits them will be silently rewritten by the cluster. Generate these correctly up front (see `/kickstart-generate`) so the mutators are no-ops. +- **Part B — Pod security & deployment best practice.** Additional hardening checks Kickstart applies. Valuable, but *not* the AKS Deployment Safeguards policy set — don't conflate the two when reporting. + +--- + +# Part A — AKS Deployment Safeguards (policy-enforced) + +| # | Safeguard policy | Severity | Mutation outcome if available | +|---|---|---|---| +| A1 | Cannot Edit Individual Nodes | HIGH | N/A — rejected, not mutated | +| A2 | Containers CPU and memory resource **requests** must be defined | HIGH | **Mutates** — sets default CPU/memory requests and enforces minimums | +| A3 | Must have anti-affinity rules or `topologySpreadConstraints` set | MEDIUM | **Mutates** — adds pod anti-affinity + topology spread constraints (multi-replica workloads) | +| A4 | No AKS-specific labels | MEDIUM | N/A | +| A5 | Containers should only use allowed images | HIGH | N/A | +| A6 | Reserved system pool taints | MEDIUM | **Mutates** — removes the `CriticalAddonsOnly` taint from user node pools | +| A7 | Containers have readiness or liveness probes configured | HIGH | N/A | +| A8 | Clusters should use CSI driver StorageClass | MEDIUM | N/A | +| A9 | Services should use unique selectors | HIGH | N/A | +| A10 | Container images should not include `latest` tag | HIGH | N/A | + +### A1: cannot-edit-individual-nodes +- **Check**: No manifest, script, or command targets an individual Node — no `kind: Node` objects, no `kubectl label/taint/cordon node`, no `nodeSelector`/`nodeName` pinning to a specific node name. Use node pools and pool-level labels instead. +- [ ] Pass / Fail + +### A2: require-requests *(mutating)* +- **Check**: Every container (including init containers and sidecars) declares `resources.requests.cpu` **and** `resources.requests.memory`. Limits alone are not sufficient. +- **If omitted**: the injected defaults are large — **CPU `500m` and memory `2048Mi` (2Gi) per container**, set as both request and limit. On a multi-container pod that is a real scheduling and cost surprise, so always declare requests explicitly. +- **Enforced minimums**: CPU `100m`, memory `100Mi`. Values below these are raised. If a request ends up above its limit the request is capped to the limit to keep the QoS class valid. +- [ ] Pass / Fail + +### A3: require-spread-or-anti-affinity *(mutating)* +- **Applies to**: multi-replica workloads. The policy error reads `Deployment with 2 replicas should have either podAntiAffinity or topologySpreadConstraints set` — single-replica workloads aren't flagged. Generate the constraints anyway, so scaling up later doesn't silently trigger a mutation. +- **Check**: Each Deployment/StatefulSet sets `spec.template.spec.topologySpreadConstraints` **or** `affinity.podAntiAffinity`. Prefer topology spread on `kubernetes.io/hostname` with `whenUnsatisfiable: ScheduleAnyway`. +- **If omitted**: AKS adds a preferred pod anti-affinity rule (weight 100, topology key `kubernetes.io/hostname`) plus a topology spread constraint (`maxSkew: 1`, `whenUnsatisfiable: ScheduleAnyway`). It picks the selector label by priority — `app`, then `app.kubernetes.io/name`, else a generated `default-antiaffinity-applabel=`. The mutator skips a workload entirely if *either* pod anti-affinity or any topology spread constraint already exists. +- [ ] Pass / Fail + +### A4: no-aks-specific-labels +- **Check**: No object sets a `kubernetes.azure.com/*` **label**. These are reserved for AKS (`Label kubernetes.azure.com is reserved for AKS use only`). +- **Scope**: labels only. The policy does not inspect `metadata.annotations` — the safeguards docs name the annotation field explicitly where they mean it (e.g. the AppArmor rule). Even so, don't put `kubernetes.azure.com/*` in an annotation; the prefix is reserved by convention regardless of which field is enforced. +- `azure.workload.identity/*` labels and annotations are **not** covered by this rule and are required for Workload Identity. +- [ ] Pass / Fail + +### A5: allowed-images-only +- **Check**: Every image resolves to a registry the cluster permits — normally the Phase 2 ACR (`.azurecr.io/...`). Flag any third-party image (`docker.io/...`, `ghcr.io/...`) that must first be brought in with `az acr import`. +- [ ] Pass / Fail + +### A6: reserved-system-pool-taints *(mutating)* +- **Applies to**: Node and node-pool configuration, not Pod tolerations. The policy evaluates node `spec.taints` and excludes system pools by their `kubernetes.azure.com/mode` label. +- **Check**: No generated user node pool config sets the `CriticalAddonsOnly` taint. AKS reserves it for system pools. +- **If present**: AKS removes the taint from the user pool. +- [ ] Pass / Fail (N/A when no user node pool configuration is generated) + +### A7: require-probes +- **Check**: Every container defines a `readinessProbe` **or** `livenessProbe` (Kickstart generates both). The path and port must match the app's real health endpoint from the structure map — not a guessed `/healthz`. +- [ ] Pass / Fail + +### A8: csi-storageclass +- **Check**: the policy evaluates the StorageClass **provisioner**, not its name. In-tree `kubernetes.io/azure-disk` and `kubernetes.io/azure-file` are rejected (`Storage class use intree provisioner ... is not allowed`); use `disk.csi.azure.com` or `file.csi.azure.com`. +- In practice: set `storageClassName` on every PVC to a CSI-backed class (`managed-csi`, `managed-csi-premium`, `azurefile-csi`) rather than relying on an unset default, and verify the class's provisioner if it's cluster-custom. +- [ ] Pass / Fail (N/A when no PVCs) + +### A9: unique-service-selectors +- **Check**: No two Services share a selector, and each Service's selector matches exactly one workload. In monorepos do not reuse `app: ` across services — use `app: ` or `app.kubernetes.io/name` + `app.kubernetes.io/component`. +- [ ] Pass / Fail + +### A10: no-latest-tag +- **Check**: No image reference ends in `:latest` or omits a tag (an untagged image resolves to `latest`). Applies to init containers and sidecars too. +- [ ] Pass / Fail + +--- + +### Enforcement caveats + +- **Gatekeeper runs fail-open.** If the admission webhook doesn't respond, validation is skipped and a non-compliant workload is admitted. Safeguards are a backstop, not a guarantee — generate compliant manifests rather than relying on enforcement to catch mistakes. +- **All or nothing.** Safeguards can't be enabled selectively; turning on `Warn` or `Enforce` activates every policy. Namespaces can be excluded, but on AKS Automatic the level can't be lowered from `Enforce`. + +--- + +# Part B — Pod security & best practice + +### Rule: no-system-pool-toleration +- **Severity**: MEDIUM +- **Description**: Ordinary app pods should not tolerate the system pool's `CriticalAddonsOnly` taint. A toleration permits scheduling onto a matching tainted node but does not attract the pod there; this is a Kickstart best practice, not the A6 policy check. +- **Check**: Verify app workloads do not declare a `CriticalAddonsOnly` toleration. +- [ ] Pass / Fail ### Rule: no-privileged - **Severity**: HIGH @@ -18,7 +104,7 @@ This skill provides a comprehensive checklist for validating generated Kubernete ### Rule: require-limits - **Severity**: MEDIUM -- **Description**: All containers must declare resource limits (CPU and memory). +- **Description**: All containers must declare resource limits (CPU and memory). Requests are covered separately by the policy-enforced **A2**. - **Check**: Verify that `spec.containers[*].resources.limits` is defined for all containers - [ ] Pass / Fail @@ -28,12 +114,6 @@ This skill provides a comprehensive checklist for validating generated Kubernete - **Check**: Verify that `spec.volumes[*].hostPath` is null or not present - [ ] Pass / Fail -### Rule: no-latest-tag -- **Severity**: HIGH -- **Description**: Container images must not use the ':latest' tag. -- **Check**: Verify that `spec.containers[*].image` does not end with `:latest` -- [ ] Pass / Fail - ### Rule: no-privilege-escalation - **Severity**: HIGH - **Description**: Containers must not allow privilege escalation. @@ -111,13 +191,22 @@ This skill provides a comprehensive checklist for validating generated Kubernete When possible, use the `runCommands` tool to validate manifests programmatically: ```bash -# Dry-run validation against K8s API schemas +# Schema-only validation (works with no cluster) kubectl apply --dry-run=client -f k8s/ +# BEST: server dry-run runs the real admission webhooks — this is what actually +# evaluates Part A, and it shows you the mutated result before you commit to it. +kubectl apply --dry-run=server -f k8s/ + +# Diff your YAML against what the cluster would store (reveals safeguard mutations) +kubectl diff -f k8s/ + # Validate with kubeconform (if installed) kubeconform -strict -summary k8s/*.yaml ``` +`--dry-run=server` requires cluster credentials (Phase 6). Before that, evaluate Part A by reading the manifests. + ## Review Instructions When reviewing manifests, use this checklist to validate each safeguard rule: @@ -130,16 +219,31 @@ When reviewing manifests, use this checklist to validate each safeguard rule: 3. **Report results** in a summary table showing rule ID, status, and any notes 4. **Block on failures**: Any FAIL on a **HIGH-severity** rule must be fixed before the manifest proceeds to deployment 5. **Address medium-severity failures**: MEDIUM-severity FAILs should be resolved or explicitly justified before proceeding +6. **Never leave a mutating safeguard (A2, A3, A6) to the cluster.** A FAIL there won't block admission — AKS will quietly rewrite the object, so the deployed state stops matching the generated YAML and later `kubectl diff` output becomes confusing. Fix these in the manifest even though they "would work anyway." +7. **Report Part A and Part B separately** so the user can see policy compliance distinctly from hardening advice. ## Example Review Output +**Part A — AKS Deployment Safeguards** + +``` +| ID | Safeguard | Severity | Mutating | Status | Notes | +|----|-----------|----------|----------|--------|-------| +| A2 | require-requests | HIGH | yes | ✓ PASS | cpu 100m / memory 128Mi on all containers | +| A3 | spread-or-anti-affinity | MEDIUM | yes | ✓ PASS | topologySpreadConstraints on hostname | +| A9 | unique-service-selectors | HIGH | no | ✗ FAIL | api and worker Services both select app: store | +| A10 | no-latest-tag | HIGH | no | ✓ PASS | pinned tag v1.2.3 | +| ... | ... | ... | ... | ... | ... | +``` + +**Part B — Pod security & best practice** + ``` | Rule ID | Severity | Status | Notes | |---------|----------|--------|-------| | no-privileged | HIGH | ✓ PASS | securityContext.privileged is false | | require-limits | MEDIUM | ✓ PASS | All containers have CPU/memory limits | | no-hostpath | HIGH | ✓ PASS | No hostPath volumes defined | -| no-latest-tag | HIGH | ✓ PASS | Image uses pinned tag v1.2.3 | | ... | ... | ... | ... | ``` @@ -152,4 +256,4 @@ When reviewing manifests, use this checklist to validate each safeguard rule: --- -*Last updated: Safeguards from `packages/pack-aks-automatic/src/safeguards.json`* +*Part A: AKS Automatic Deployment Safeguards policy set (mutation outcomes per AKS docs). Part B: `packages/pack-aks-automatic/src/safeguards.json`.*