Skip to content

perf(tfork): inject source files while frozen - #13

Open
yiying-zhang wants to merge 70 commits into
mainfrom
perf/tfork-inject-source-file
Open

perf(tfork): inject source files while frozen#13
yiying-zhang wants to merge 70 commits into
mainfrom
perf/tfork-inject-source-file

Conversation

@yiying-zhang

Copy link
Copy Markdown
Collaborator

What this changes

This PR adds a repeatable live-clone option:

--tfork-inject-source-file=HOST_PATH:CONTAINER_PATH

It rotates small files in the live source after clone filesystem snapshots have
been created. Children therefore retain the pre-rotation filesystem state while
the source resumes with the new files.

The option is intended for source-side credential or context rotation. A
successfully committed rotation is durable: if later clone publication fails,
Podman rolls back the unpublished children but deliberately does not overwrite
files that the live source may already have observed.

Ordering and commit semantics

When source injection is requested, Podman now:

  1. parses every injection and preloads every host payload before freezing the
    source;
  2. freezes the source and creates all clone rootfs snapshots;
  3. completes CRIU restore successfully while keeping the source frozen;
  4. stages every replacement as a temporary regular file in its destination
    directory;
  5. commits the staged batch while the source is still frozen;
  6. thaws the source and continues normal clone publication.

The normal no-injection path still thaws at CRIU's source-detached event. Only
source-rotation requests defer that early thaw until restore and rotation finish.

The rotation commit is the side effect's explicit point of no return:

  • failures before or during the rotation restore all previously existing source
    files, remove newly created destinations, and remove staging files;
  • failures after a successful rotation retain the new source files, roll back
    unpublished clones, and annotate the returned error to say that the source
    rotation was not rolled back.

Rolling the files back after source thaw would be unsafe because the live source
may already have read or modified them.

Atomic installation

All host sources are opened with O_NOFOLLOW, validated as regular files no
larger than 1 MiB, and read completely before source mutation begins. Duplicate
container destinations are rejected.

For each destination, Podman securely opens the parent beneath
/proc/<source-pid>/root, writes a same-directory temporary file, and then uses
renameat to install it. Existing files are first renamed to private backups so
the complete batch can be restored in reverse order if any commit step fails.

This prevents the source from being thawed with a truncated file, a partial
copy, or only part of a multi-file rotation. Existing destination ownership is
preserved; newly created files inherit Podman's filesystem UID/GID; all installed
files use mode 0600.

The batch is failure-atomic for errors handled by Podman. It is not a filesystem
transaction that is crash-atomic across multiple destination directories.

Performance impact

Forks without --tfork-inject-source-file keep the existing early-thaw fast
path unchanged.

When rotation is requested, source pause time additionally includes the tail of
CRIU restore plus staging and renaming the requested files. Payloads remain
limited to 1 MiB each, and host reads occur before the source is frozen.

Validation

  • source-injection parser coverage, including duplicate destinations;
  • successful multi-file commit tests;
  • injected mid-commit failure tests verifying restoration of existing files,
    removal of newly created files, and cleanup of temporary/backup files;
  • go test ./cmd/podman/containers ./pkg/domain/infra/abi;
  • targeted source-rotation tests under the Go race detector;
  • go vet ./cmd/podman/containers ./pkg/domain/infra/abi;
  • existing real-Webtop scaling validation: 50 forks with exact child/source
    process validation and no D-state leakage after a readiness timeout.

yiying-zhang and others added 30 commits July 10, 2026 23:44
Stabilize live tfork source freezing
yiying-zhang and others added 30 commits July 20, 2026 03:55
…-main

Stabilize single-copy tfork clone path
Rewrite tclone setup for Gensee Crate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants