Add Client API Cell protocol vocabulary (client/cell/defs.py) by YuanTingHsieh · Pull Request #4856 · NVIDIA/NVFlare

YuanTingHsieh · 2026-07-02T16:40:12Z

What

New pure-library module nvflare/client/cell/defs.py — the Client API Cell control-protocol vocabulary, consumed by the trainer-side Cell engine (TE-3), external_process (EP-3), and attach (AT-2) so the wire contract is defined once.

CHANNEL ("client_api"), PROTOCOL_VERSION.
Topic — HELLO / HELLO_CHALLENGE / HELLO_PROOF / HELLO_ACCEPTED / HELLO_REJECTED, TASK_READY / TASK_ACCEPTED / TASK_FAILED, RESULT_READY / RESULT_ACCEPTED / RESULT_REJECTED, LOG, HEARTBEAT, ABORT, SHUTDOWN, BYE, ERROR (values client_api.-prefixed to avoid colliding with the legacy flare_agent topics).
MsgKey payload keys.

Pure constants — no Cell/cellnet imports, no I/O. Applies to the out-of-process modes only; in_process uses DataBus and does not use this protocol.

Scope change (this PR was trimmed)

The auth mechanism (auth.py — token generation + HMAC challenge-response proof) was removed from this freeze. Rationale: the token is three roles — rendezvous, anti-mixup, and authentication — and whether external_process needs a secret HMAC proof at all is a host-trust decision (single-tenant: OS isolation suffices; multi-tenant: needed) that belongs to EP-3, not to a module frozen ahead of it. Freezing the vocabulary (undisputed, shared) is right; freezing an auth mechanism before its model is decided is not.

Where the auth pieces now live:

proof helpers (TokenScope, compute/verify_hello_proof, combine_nonces) → EP-3 (owns the auth-model decision, first consumer)
generic generators (generate_session_token/generate_nonce, token_digest) → FLARE-3017 (fuel/sec consolidation)
stateful SessionTokenManager → AT-2 (attach-only)

(The removed code is preserved in this branch's history for EP-3 to draw on.)

Program context

Client API Execution Modes (2.9) — TE-1, Wave 0, interface freeze #1 (Epic FLARE-2698 / Story FLARE-3011).
Design: docs/design/client_api_execution_modes.md § "Control Protocol".
Unblocks: TE-3 (trainer engine), EP-3 (external_process HELLO + auth model), AT-2 (attach).

Testing

9 unit tests pin the exact frozen wire values (CHANNEL, every Topic string, every MsgKey) so a value rename fails CI — the strings are the cross-track contract. black/isort/flake8 clean.

🤖 Generated with Claude Code

…ives TE-1 (Wave 0, interface freeze #1) of the Client API Execution Modes program (see docs/design/client_api_execution_modes_plan.md on PR NVIDIA#4853). New pure-library package nvflare/client/cell consumed by the trainer engine (TE-3), external_process (EP-3), and attach (AT-2) tracks so the HELLO handshake is implemented once: - defs.py: the Client API control-protocol vocabulary - channel, PROTOCOL_VERSION, Topic (HELLO/HELLO_CHALLENGE/HELLO_PROOF/ HELLO_ACCEPTED, TASK_READY/TASK_ACCEPTED/TASK_FAILED, RESULT_READY/ RESULT_ACCEPTED/RESULT_REJECTED, LOG, HEARTBEAT, ABORT, SHUTDOWN, BYE, ERROR) and MsgKey payload keys. Topic values are "client_api."-prefixed to avoid colliding with ipc/defs.py values. - auth.py: session-token primitives per the design's attach auth contract - TokenScope (frozen, binds token to job/site/attach id/ target FQCN/trainer FQCN/rank policy/protocol version), generate_session_token/generate_nonce (secrets), token_digest (sha256; anything persisted stores only the digest), compute/verify_hello_proof (HMAC-SHA256 keyed by the token over a domain-tagged, length-prefixed serialization of nonce + full scope; constant-time compare), and SessionTokenManager (single-use nonces consumed on any attempt, attach_timeout expiry with injectable clock, single-session enforcement, invalidate; thread-safe; no cellnet/file-I/O deps). 37 unit tests: proof round-trip, tamper/replay/consumed-nonce/scope- mismatch/expiry/single-session/invalidation matrices, serialization non-ambiguity, and vocabulary uniqueness incl. non-collision with ipc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR introduces a new pure-library nvflare.client.cell package that centralizes the Client API control-protocol vocabulary (defs.py) and session-token based HELLO handshake/auth primitives (auth.py), along with unit tests that lock down the cross-track wire contract.

Changes:

Added frozen protocol/channel/topic/message-key constants for the Client API control protocol (nvflare/client/cell/defs.py).
Added session-token auth primitives (token scope, proof compute/verify, nonce issuance/bounding, attach-timeout + invalidation) in a Cell-independent library (nvflare/client/cell/auth.py).
Added unit tests asserting exact wire strings and extensive proof/nonce/session-manager security properties (tests/unit_test/client/cell/*).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
nvflare/client/cell/defs.py	Adds the frozen channel/topic/message-key vocabulary for the Client API control protocol.
nvflare/client/cell/auth.py	Adds session-token / HELLO proof primitives and an executor-side token+nonce manager.
nvflare/client/cell/init.py	Introduces the new `nvflare.client.cell` package.
tests/unit_test/client/cell/defs_test.py	Locks down exact wire values and collision avoidance with legacy IPC defs.
tests/unit_test/client/cell/auth_test.py	Comprehensive unit tests for proof integrity, replay resistance, expiry, invalidation, and bounded nonce behavior.
tests/unit_test/client/cell/init.py	Introduces the test package namespace for `tests.unit_test.client.cell`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov-commenter · 2026-07-02T16:48:24Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.27%. Comparing base (c0efa72) to head (a91691a).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4856      +/-   ##
==========================================
+ Coverage   60.24%   60.27%   +0.02%     
==========================================
  Files         970      971       +1     
  Lines       92423    92463      +40     
==========================================
+ Hits        55682    55731      +49     
+ Misses      36741    36732       -9

Flag	Coverage Δ
unit-tests	`60.27% <100.00%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

greptile-apps · 2026-07-02T16:49:03Z

Greptile Summary

This PR introduces nvflare/client/cell/defs.py, a pure-constants vocabulary module defining the Client API Cell control-protocol wire contract (channel name, protocol version, Topic and MsgKey classes). Nine unit tests pin every wire string value so that any rename breaks CI.

nvflare/client/cell/defs.py: Adds CHANNEL, PROTOCOL_VERSION, Topic (17 topic constants including the newly-added HELLO_REJECTED), and MsgKey (19 payload-key constants). No imports, no I/O — pure vocabulary.
tests/unit_test/client/cell/defs_test.py: 9 tests verify exact wire values, uniqueness, absence of collisions with the legacy flare_agent channel/topics, and the complete set of topic names.

Confidence Score: 5/5

Safe to merge — pure constants module with no logic, no runtime dependencies, and comprehensive wire-value freeze tests.

The change is a vocabulary-only module: two module-level constants and two classes of string literals. There is no logic that can break, no I/O, and no imports beyond what the test file adds. All 17 topic strings and 19 message-key strings are pinned by exact-equality assertions, so any future accidental rename will fail CI before it reaches consumers.

No files require special attention.

Important Files Changed

Filename	Overview
nvflare/client/cell/defs.py	New pure-constants protocol vocabulary module; no imports, no logic, well-documented. Wire values are correct and consistently namespaced under `client_api.`.
tests/unit_test/client/cell/defs_test.py	Comprehensive contract-freeze tests: pins exact wire strings for every Topic and MsgKey, checks uniqueness, collision guard against legacy ipc defs, and topic-set membership. No issues.
nvflare/client/cell/init.py	New package marker — license header only, no code.
tests/unit_test/client/cell/init.py	New test package marker — license header only, no code.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant T as Trainer (TE-3)
    participant S as Server / FL Engine

    T->>S: HELLO (SESSION_ID, JOB_ID, SITE_NAME, TRAINER_FQCN, RANK, PROTOCOL_VERSION)
    S->>T: HELLO_CHALLENGE (NONCE)
    T->>S: HELLO_PROOF (PROOF, SESSION_ID)
    alt Auth accepted
        S->>T: HELLO_ACCEPTED
    else Auth refused
        S->>T: HELLO_REJECTED (REASON)
    end

    loop Each training round
        T->>S: TASK_READY (SESSION_ID)
        S->>T: TASK_ACCEPTED (TASK_ID, TASK_NAME, MODEL, PARAMS)
        Note over T: Training...
        T->>S: RESULT_READY (RESULT_ID, MANIFEST, TRANSFER_ID)
        alt Result OK
            S->>T: RESULT_ACCEPTED
        else Result bad
            S->>T: RESULT_REJECTED (REASON)
        end
        T-->>S: HEARTBEAT (periodic)
        T-->>S: LOG (periodic)
    end

    alt Normal teardown
        S->>T: SHUTDOWN
        T->>S: BYE
    else Abort
        S->>T: ABORT
    else Protocol fault
        S->>T: ERROR (REASON)
    end

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant T as Trainer (TE-3)
    participant S as Server / FL Engine

    T->>S: HELLO (SESSION_ID, JOB_ID, SITE_NAME, TRAINER_FQCN, RANK, PROTOCOL_VERSION)
    S->>T: HELLO_CHALLENGE (NONCE)
    T->>S: HELLO_PROOF (PROOF, SESSION_ID)
    alt Auth accepted
        S->>T: HELLO_ACCEPTED
    else Auth refused
        S->>T: HELLO_REJECTED (REASON)
    end

    loop Each training round
        T->>S: TASK_READY (SESSION_ID)
        S->>T: TASK_ACCEPTED (TASK_ID, TASK_NAME, MODEL, PARAMS)
        Note over T: Training...
        T->>S: RESULT_READY (RESULT_ID, MANIFEST, TRANSFER_ID)
        alt Result OK
            S->>T: RESULT_ACCEPTED
        else Result bad
            S->>T: RESULT_REJECTED (REASON)
        end
        T-->>S: HEARTBEAT (periodic)
        T-->>S: LOG (periodic)
    end

    alt Normal teardown
        S->>T: SHUTDOWN
        T->>S: BYE
    else Abort
        S->>T: ABORT
    else Protocol fault
        S->>T: ERROR (REASON)
    end

_{Reviews (5): Last reviewed commit: "Merge branch 'main' into yuantingh/clien..." | Re-trigger Greptile}

…ttach Scope refinement (PR NVIDIA#4856): external_process — the primary mode, which shells out to torchrun/Deepspeed/Horovod/mpirun and talks only to rank 0 over a localhost connection NVFlare itself created — needs only a lightweight launch-token proof. The stateful executor-side SessionTokenManager (single-use nonce issuance, attach-window expiry, single-session enforcement, invalidation) is a challenge-response concern specific to attach mode, where NVFlare does not own the trainer process and the token is delivered out-of-band by an untrusted starter. Move SessionTokenManager to the attach backend (AT-2) so this freeze ships only what P0 (external_process) needs: the defs vocabulary and the stateless primitives (TokenScope, generate_session_token/generate_nonce, token_digest, compute_hello_proof/verify_hello_proof, combine_nonces). 30 tests remain (the manager's tests move with it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…CTED PR NVIDIA#4856 review fixes: - verify_hello_proof rejects any proof whose length is not the fixed SHA-256 hex length (PROOF_HEX_LEN=64) up front, so an arbitrarily long attacker proof cannot force HMAC/compare work; also guards AttributeError so a non-str token/nonce (e.g. a None token from a config miss on the one-round path) returns False instead of raising. - generate_session_token / generate_nonce reject below a 128-bit entropy floor (_MIN_ENTROPY_BYTES=16) so a public primitive can't mint an empty or weak token/nonce (num_bytes=0). - add Topic.HELLO_REJECTED so a handshake auth refusal (bad proof, wrong scope, consumed/expired nonce, single-session) is a distinct semantic signal from the generic ERROR (protocol/transport fault) for TE-3/AT-2. Tests cover the entropy floors, wrong/huge proof lengths, non-str token/nonce, a 64-char non-ASCII proof reaching compare_digest, and the HELLO_REJECTED wire value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Trim auth.py (and its tests) out of the interface freeze. The token is three roles — rendezvous, anti-mixup, and authentication — and whether external_process needs a *secret* HMAC proof at all is a host-trust decision (single-tenant: OS isolation suffices; multi-tenant: needed) that belongs to EP-3, not to a frozen module shipped ahead of it. NVIDIA#4856 now freezes only what is undisputed and shared: the Cell control- protocol vocabulary (client/cell/defs.py — Topics, MsgKeys, CHANNEL, PROTOCOL_VERSION). The proof helpers (TokenScope, compute/verify_hello_proof, combine_nonces) move to EP-3 where the auth model is decided and first consumed; the generic generators (generate_session_token/nonce, token_digest) go to the fuel/sec consolidation (FLARE-3017); the stateful SessionTokenManager remains attach-only (AT-2). Plan updated to match. The removed code is preserved in this branch's history for EP-3 to draw on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Reflect the NVIDIA#4856 trim: TE-1 (interface freeze #1) is now the Cell control-protocol vocabulary only. The auth mechanism is not frozen ahead of its decision — the proof helpers land with EP-3 (which owns the host-trust / auth-strength call: rendezvous-only vs one-round HMAC), the generic token/nonce/digest generators go to FLARE-3017, and the stateful SessionTokenManager stays attach-only (AT-2). Updated the TE-1, EP-3, AT-2 rows and the interface-freeze note accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings July 2, 2026 16:40

Copilot started reviewing on behalf of YuanTingHsieh July 2, 2026 16:40 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Comment thread nvflare/client/cell/auth.py Outdated

Comment thread nvflare/client/cell/auth.py Outdated

Comment thread nvflare/client/cell/auth.py Outdated

greptile-apps Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread nvflare/client/cell/auth.py Outdated

Comment thread nvflare/client/cell/defs.py

Comment thread nvflare/client/cell/auth.py Outdated

YuanTingHsieh changed the title ~~Add Client API Cell protocol vocabulary and session-token auth primitives~~ Add Client API Cell protocol vocabulary and stateless session-token proof primitives Jul 2, 2026

YuanTingHsieh and others added 2 commits July 2, 2026 14:20

YuanTingHsieh changed the title ~~Add Client API Cell protocol vocabulary and stateless session-token proof primitives~~ Add Client API Cell protocol vocabulary (client/cell/defs.py) Jul 2, 2026

pcnudde approved these changes Jul 7, 2026

View reviewed changes

Merge branch 'main' into yuantingh/client-api-te1-protocol-auth

a91691a

YuanTingHsieh merged commit 6be8afc into NVIDIA:main Jul 7, 2026
18 checks passed

YuanTingHsieh deleted the yuantingh/client-api-te1-protocol-auth branch July 7, 2026 22:57

YuanTingHsieh mentioned this pull request Jul 7, 2026

Client API design rev 2.2: heartbeat phases, per-launch tokens, F3 dependencies; add TASK_PAYLOAD_READY #4866

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add Client API Cell protocol vocabulary (client/cell/defs.py)#4856

Add Client API Cell protocol vocabulary (client/cell/defs.py)#4856
YuanTingHsieh merged 5 commits into
NVIDIA:mainfrom
YuanTingHsieh:yuantingh/client-api-te1-protocol-auth

YuanTingHsieh commented Jul 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jul 2, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

YuanTingHsieh commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Scope change (this PR was trimmed)

Program context

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

YuanTingHsieh commented Jul 2, 2026 •

edited

Loading

codecov-commenter commented Jul 2, 2026 •

edited

Loading

greptile-apps Bot commented Jul 2, 2026 •

edited

Loading