Skip to content

devenv machines - #3073

Draft
domenkozar wants to merge 3 commits into
mainfrom
feature/machines-secretspec
Draft

domenkozar wants to merge 3 commits into
mainfrom
feature/machines-secretspec

Conversation

@domenkozar

Copy link
Copy Markdown
Member

What this adds

This introduces an experimental machines interface for managing NixOS, nix-darwin, and home-manager configurations from the same devenv project that contains the development environment.

Users can now:

  • describe machines next to their project configuration;
  • inspect and build machine closures without touching a host;
  • install fresh NixOS systems over SSH using kexec, disko, and nixos-facter;
  • deploy updates to NixOS, nix-darwin, and local or remote home-manager targets;
  • deploy or install several machines concurrently, with a combined result summary;
  • use compatible machines as remote Nix builders; and
  • bootstrap files such as sops-nix or agenix identities through SecretSpec.

The interface is documented as experimental so it can evolve based on real-world use.

Define, inspect, build, and deploy a machine

A machine lives in devenv.nix and can contain one or more roles:

{ ... }: {
  machines.web = {
    system = "x86_64-linux";
    target.host = "root@web.example.com";

    nixos = {
      networking.hostName = "web";
      services.openssh.enable = true;
    };

    home-manager = {
      home.username = "deploy";
      home.homeDirectory = "/home/deploy";
      programs.git.enable = true;
    };
  };
}

Users can inspect or build it without connecting to the target:

$ devenv machines info
$ devenv build machines.web
$ devenv build machines.web.build.nixos

Once the host is running, one command builds, copies, and activates its declared roles in the correct order:

$ devenv machines deploy web

NixOS activates first and home-manager follows, so a system configuration can create a user before its home configuration is applied. nix-darwin uses the same model, while home-manager may also be activated locally by omitting target.host.

Running devenv machines deploy without names deploys all remote targets. Independent machines run in parallel, failures are collected into a final summary, and --max-concurrent N controls rollout concurrency.

Install a fresh NixOS host

With a disko layout declared under the NixOS module, devenv can provision a machine that only has SSH and a Linux kernel:

$ devenv machines install web

The install pipeline:

  1. checks the target;
  2. kexecs into a minimal NixOS installer;
  3. probes hardware with nixos-facter and writes .machines/web/facter.json;
  4. partitions and formats disks with disko;
  5. copies and installs the NixOS closure;
  6. installs bootstrap files and preserves SSH host keys when configured; and
  7. reboots.

Install always requires explicit machine names because it can wipe disks. Multiple named hosts may be installed together, and flags allow operators to select phases, stop after disko, skip reboot, or limit concurrency.

The generated facter report is intended to be committed, allowing teammates and CI to reproduce the machine closure without probing the live host.

Bootstrap runtime secret managers with SecretSpec

Runtime secrets should still be managed by tools such as sops-nix or agenix. This feature handles the initial credential those tools need on first boot.

Declare the SecretSpec entry normally:

[project]
name = "infrastructure"
revision = "1.0"
require_reason = false

[profiles.production]
WEB_AGE_KEY = { description = "sops age identity for web" }

Then map it to a file in the installed system:

machines.web = {
  target.host = "root@web.example.com";

  install.secrets."/var/lib/sops-nix/key.txt" = {
    secret = "WEB_AGE_KEY";
    owner = "0:0";
    mode = "0600";
  };

  nixos.sops.age.keyFile = "/var/lib/sops-nix/key.txt";
};

Resolve on the operator's machine

This is the default. The selected SecretSpec provider resolves the value locally, and devenv streams only its bytes over authenticated SSH.

Resolved values are kept out of Nix evaluation, store paths, process arguments, and generated remote scripts. Transfers use private temporary files, byte-counted framing, strict modes, sync, and atomic rename. Installs carrying local payloads require a pre-trusted SSH host key and disable forwarding features.

This mode works well when an operator already has provider access and is intentionally provisioning a machine.

Resolve on the target

For environments where the operator's machine should never receive the provider credential or secret value:

machines.web.install.secretspec = {
  execution = "target";
  profile = "production";
  extraPackages = targetPkgs: [ targetPkgs.sops ];
};

In this mode, devenv sends a self-contained SecretSpec declaration manifest, not fetched values or provider credentials. The live installer resolves only the requested entries using credentials already available to that target—for example workload identity, instance metadata, or target-side SecretSpec configuration.

The resolver is the SecretSpec binary bundled with the same devenv release and built for the target architecture. Provider helper tools can be added to its private runtime with extraPackages; they do not become global system packages.

This preserves SecretSpec's provider freedom: manifests, aliases, profiles, references, scopes, validation policy, environment selection, and global target configuration continue to work normally. Provider and profile overrides are optional rather than imposed by devenv.

Other useful workflows

  • Deploy mixed NixOS, nix-darwin, and home-manager fleets from one project.
  • Combine a system role and home-manager role on one target.
  • Use --use-machines-as-builders when the operator cannot build a target architecture locally.
  • Build closures in CI or populate a cache with devenv build machines.<name> without deploying.
  • Bootstrap LUKS inputs and additional install-time files without embedding them in the Nix store.
  • Preserve existing SSH host keys across a fresh NixOS installation.

Supporting inputs such as disko, nixos-facter, nix-darwin, and home-manager are lazy: users only need the inputs for roles they actually configure. The former configurations option remains available through a rename compatibility module.

Documentation

Adds a complete Machines guide covering:

  • NixOS installation and updates;
  • disko safety and hardware reports;
  • nix-darwin and home-manager deployment;
  • multi-role and multi-machine behavior;
  • local and target-side SecretSpec trust models;
  • SSH defaults and secure file transfer;
  • remote builders, cross-platform deployments, and troubleshooting; and
  • known limitations and recovery behavior.

The generated option reference includes the new machine configuration surface.

Validation

  • nix build .#devenv -L
  • packaged devenv 2.2.2+5016975 exposes the new Machines CLI and bundled SecretSpec 0.17.1
  • 33 focused Rust machine/backend/security tests
  • Machines CLI integration tests against the freshly packaged binary
  • NixOS closure integration test proving the target-architecture closure contains and executes the bundled SecretSpec resolver
  • formatting, shell syntax, and git diff --check
  • all remaining workspace crates and doctests pass when excluding two unrelated failures in unchanged code

The full workspace run currently encounters two pre-existing/unrelated test failures in unchanged files:

  • devenv-shell::status_line_cache_invalidates_for_state_width_and_spinner_changes, whose elapsed-time assertion can cross a millisecond boundary;
  • devenv-tasks::concurrent_dynamic_starts_launch_each_process_and_oneshot_once, which passed on isolated rerun.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🔍 Suggested Reviewers

Based on git blame analysis of the changed lines, the following contributors have significant experience with the modified code:

  • @sandydoo - 95.5% of changed lines (21 lines)

Please consider reviewing this PR as you have authored significant portions of the code being modified. Your expertise would be valuable! 🙏

This comment was automatically generated by git-blame-auto-reviewer

Last updated: 2026-08-30T19:11:23.821Z

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 7, 2026

Copy link
Copy Markdown

Deploying devenv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 06b2f4c
Status: ✅  Deploy successful!
Preview URL: https://9e3731dc.devenv.pages.dev
Branch Preview URL: https://feature-machines-secretspec.devenv.pages.dev

View logs

@domenkozar
domenkozar force-pushed the feature/machines-secretspec branch 3 times, most recently from 5e6291a to ed61d24 Compare August 22, 2026 15:23
Add machine build targets, concurrent deploy/install orchestration, SecretSpec bootstrap support, remote builder integration, safety checks, documentation, and integration coverage.
@domenkozar
domenkozar force-pushed the feature/machines-secretspec branch from 341db4c to 90cd42f Compare August 30, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant