Skip to content

agent: fail closed and hand persistent volumes to CDH - #1

Open
chalabi2 wants to merge 8 commits into
akash-network:akashfrom
chalabi2:jc/blackwell-e2e
Open

chalabi2 wants to merge 8 commits into
akash-network:akashfrom
chalabi2:jc/blackwell-e2e

Conversation

@chalabi2

@chalabi2 chalabi2 commented Aug 1, 2026 •

Copy link
Copy Markdown

Why

Kata must fail closed when confidential secret resolution fails and must hand measured persistent-volume descriptors to CDH without exposing keys on the host or kernel command line.

What changed

  • propagate sealed environment and sealed-mount failures from container creation
  • reject sealed environment values when CDH is unavailable
  • materialize a 0600 per-container secure-volume descriptor from measured initdata
  • require exact container and reserved OCI Block-device matches
  • delegate signed-reference verification and persistent LUKS handling to CDH
  • pin guest-components to the crash-safe persistent-volume revision
  • package blkid from util-linux in Ubuntu and NVIDIA guest root filesystems
  • allow only measured cc_init_data for NVIDIA SNP and TDX confidential runtime entries
  • retain both CDI and VFIO devices in two-pGPU regression tests
  • preserve ordinary environment values and ordinary mounted files

Dependencies

Validation

  • secure-volume descriptor, exact-device, initdata materialization, and permission tests pass
  • missing-CDH sealed-environment regression passes
  • Rust formatting and Clippy with warnings denied pass
  • focused two-VFIO and two-pGPU Go tests pass
  • Helm rendering emits measured initdata for the confidential GPU shims
  • current Linux static checks compile the changed agent, runtime, and packaging paths
  • shellcheck, spelling, editorconfig, cargo-deny, code generation, and policy coverage checks pass

Live result and limits

This branch was used in the isolated RTX PRO 6000 canary that attached two GPUs to one SNP guest and reopened persistent Block PVCs across fresh Kata VMs.

The newly pinned crash-safety commit still needs its ignored privileged loopback test on disposable Linux storage. B200 and NVSwitch remain separate hardware claims. The QMP pxb-host.addr warnings observed during the canary did not prevent either VFIO device from attaching and are not addressed here.

The PR still has repository-infrastructure failures unrelated to the changed code: the fork lacks the ok-to-test gate, static checks cannot upload an organization ORAS cache, and existing workflow files trigger actionlint/zizmor.

chalabi2 added 2 commits July 31, 2026 22:29
Propagate Confidential Data Hub failures while resolving declared sealed environment values and candidate secret mounts.

This prevents container creation from continuing with sealed ciphertext or unavailable plaintext.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Lock the existing multi-device behavior at the CDI and VFIO cold-plug boundaries.

The tests prove two requested pGPU IDs remain two guest devices without changing the production passthrough path.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Format the two existing agent test expressions exactly as required by rustfmt so the branch-wide make check can run.

This does not change runtime behavior.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
@chalabi2 chalabi2 changed the title agent: fail closed on sealed secrets and cover two-pGPU agent: fail closed and hand persistent volumes to CDH Aug 1, 2026
Materialize a measured per-container volume descriptor from initdata.
Require an exact match with reserved OCI block devices before calling the
CDH persistent secure-mount API.

Remove the Provider-controlled kernel-command-line DEK path and the
duplicate cryptsetup implementation. Pin the guest-components revision
that verifies content-bound sealed keys and owns the non-destructive
LUKS2 state machine.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Abort container creation when a sealed environment value is present but the Confidential Data Hub client is unavailable.

Ordinary environments remain compatible with non-confidential Kata guests.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Permit the cc_init_data annotation for NVIDIA SNP and TDX runtimes so per-workload CDH and policy configuration reaches the guest.

Keep the allowlist limited to measured confidential-computing initdata.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Kata confidential guests use a pre-6.6 kernel that rejects the previous
image-rs fchmodat call.

Pin the guest-components backport so confidential OCI image extraction reaches
container startup.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
Pin guest-components to the crash-safe persistent LUKS revision.

Install blkid from util-linux in Ubuntu root filesystems.

Copy it into the NVIDIA runtime closure so CDH can classify mapped devices before mounting or formatting them.

Signed-off-by: Joseph Chalabi <chalabi.joseph@gmail.com>
@chalabi2
chalabi2 marked this pull request as ready for review August 4, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant