Add Auto-FL agent skill

docs/design/autofl_skill.md

@@ -0,0 +1,148 @@
+# NVFlare Auto-FL Skill Design
+
+## Summary
+
+Auto-FL should enter NVFlare as a skill-first product experience. Users select
+an official NVFlare Auto-FL skill in a coding agent, point it at an existing
+`job.py`, and state the optimization objective, environment, and budget. NVFlare
+owns deterministic import of campaign-relevant settings, execution truth, policy
+boundaries, artifacts, and reproducibility. The agent owns candidate planning,
+code edits within allowed paths, experiment execution through the existing
+`job.py`, comparison, and narrative reporting.
+
+This avoids introducing a new public Auto-FL command tree while still making
+Auto-FL an NVFlare-owned feature.
+
+## Product Boundary
+
+The first production-oriented slice includes:
+
+- A root `skills/nvflare-autofl` agent skill that follows the NVFLARE skills
+  layout used by the general agent-skills work.
+- A deterministic `job.py` importer that emits reviewable `autofl.yaml` for the
+  Auto-FL campaign.
+- A trust contract in `autofl.yaml` showing editable campaign settings,
+  unresolved fields, fixed-budget constraints, and allowed edit paths.
+- A skill-local candidate lifecycle that snapshots the current best source,
+  gives the agent an isolated draft, validates the resulting patch, and keeps or
+  restores source according to the campaign metric.
+- Documentation for using the skill with simulation, POC, and production
+  environments through existing NVFlare surfaces.
+
+The first version does not embed or vendor a coding agent, and it does not add a
+public Auto-FL command family.
+
+## Role of autofl.yaml
+
+`autofl.yaml` is not a replacement for `job.py` and is not a second exported
+job format.  The original `job.py` remains the experiment entry point the agent
+uses to run candidates, and exported job folders remain the NVFlare execution
+and submission artifacts.
+
+The purpose of `autofl.yaml` is to expose the human-reviewable Auto-FL campaign
+layer:
+
+- Objective metric, requested environment, and candidate budget.
+- Editable search-space settings discovered from `job.py` and related train
+  scripts.
+- Fixed-budget constraints that must remain comparable across candidates.
+- Allowed edit paths and files that are out of scope for the agent.
+- Allowed creation patterns for new Python modules under the job root.
+- Artifact, ledger, and report locations for the campaign.
+- Provenance and unresolved fields that need user review before safe execution.
+
+By default, users should not need to edit `autofl.yaml`.  They review or modify
+it only when the importer surfaces unresolved settings or when they want to
+override campaign knobs explicitly.
+
+## Deterministic Import
+
+The importer parses Python source with `ast`; it does not import or execute
+user code.  It supports known Recipe and FedJob-style patterns first and focuses
+on campaign-relevant settings rather than duplicating the full exported job:
+
+- Recipe/FedJob constructor and class import.
+- `SimEnv`, `PocEnv`, and `ProdEnv` references.
+- `train_script` resolution for literal and argparse-derived values.
+- Objective metric from user request, `key_metric`, or explicit unresolved
+  default.
+- Fixed-budget fields such as rounds, clients, and candidate budget.
+- Common argparse tunables from `job.py` and the resolved train script.
+
+The exported job folder remains useful as execution truth once the job is
+materialized, because it contains resolved NVFlare app and component configs.
+However, it does not reliably preserve all authoring intent needed for an
+Auto-FL campaign, such as editable source files, train-argument construction,
+tunable-versus-fixed intent, and source provenance.  Therefore the importer uses
+deterministic Python/static parsing for the campaign layer and may use exported
+config inspection as a validation aid when available.
+
+Unsupported or dynamic fields are carried forward as unresolved review items
+instead of being guessed by the importer or the agent.
+
+## Trust Contract
+
+Every import result includes:
+
+- `import`: importer version, source path, source hash, support status, and
+  confidence.
+- `job`: surface, entrypoint, allowed edit paths, train script, and call
+  arguments with provenance.
+- `objective`, `budget`, `environment`, and `search_space`.
+- `trust_contract`: extracted facts, unresolved fields, allowed edit paths, and
+  agent controls.
+
+The skill must present editable, unresolved, and allowed sections before it runs
+candidates. This is the core product guardrail: NVFlare makes the campaign
+reviewable and reproducible; the agent makes it interactive and exploratory.
+
+## Candidate Contract
+
+The agent, rather than the deterministic runner, owns search policy. It may
+change tunables, edit the imported job's allowed source files, or implement new
+algorithms as Python modules. Each attempt starts from the retained best source
+in `.nvflare/autofl/candidates/<id>/source` and has a generated
+`candidate_manifest.json` containing its hypothesis, base candidate, run
+arguments, changed files, source and budget hashes, patch hash, artifacts, and
+result.
+
+NVFlare computes the manifest's evidence fields; the agent does not assert them.
+Before execution, the helper rejects stale candidates, path traversal, symlink
+escapes, unauthorized existing-file edits, and detectable fixed-budget drift.
+It applies the candidate transactionally to the real job workspace, retains a
+new best, and restores the previous best after a discard or crash. This works
+without requiring a Git repository and leaves the best source ready for the
+standard NVFlare job lifecycle.
+
+The built-in parameter candidates are suggestion seeds only. They are returned
+as machine-readable hypotheses and arguments when requested, but are not the
+default search loop and are never executed without agent selection.
+
+## Execution Model
+
+The skill uses existing NVFlare execution surfaces:
+
+- Simulation: initialize a baseline, prepare an agent-authored candidate draft,
+  and evaluate it through the existing `job.py` and configured `SimEnv`.
+- POC: use the existing job authoring/export flow, startup kits, and standard
+  `nvflare job` commands, then record the job ID, artifacts, and metric against
+  the candidate manifest.
+- Production: use standard startup-kit authentication, site policy, job submit,
+  wait, download, and inspection commands with the same manifest and result
+  recording contract.
+
+Production is a valid optimization environment. The best candidate may later be
+submitted or reused through the standard NVFlare job lifecycle; no separate
+promotion command is needed.
+
+## Review Questions
+
+- Are the supported `job.py` patterns sufficient for an initial prototype?
+- Are the edit and creation permissions in `autofl.yaml` appropriate for
+  algorithm-level candidates while preserving candidate comparability?
+- Which exported-job fields should be used as validation evidence versus static
+  `job.py` parsing for authoring intent?
+- Does the Auto-FL skill pass the general NVFLARE skill frontmatter, trigger,
+  and eval checks after it lands under `skills/nvflare-autofl`?
+- Which candidate-manifest and metric/artifact fields should become stable
+  NVFlare APIs after the skill-local contract proves itself?

docs/user_guide/nvflare_cli/autofl_skill.rst

@@ -0,0 +1,129 @@
+.. _autofl_skill:
+
+#######################
+NVFlare Auto-FL Skill
+#######################
+
+The NVFlare Auto-FL skill is an agent-assisted workflow for optimizing an
+existing NVFlare ``job.py``.  The user entry point is the coding agent skill:
+select the NVFlare Auto-FL skill, point it at a job, and state the objective,
+environment, and candidate budget.
+
+The skill source lives in ``skills/nvflare-autofl`` with the other NVFlare-owned
+agent skills.  When the general agent skill CLI is available, install it through
+the standard ``nvflare agent skills`` workflow for the target coding agent.
+
+NVFlare does not add a separate public Auto-FL command family for this workflow.
+Instead, NVFlare provides the deterministic import, reviewable
+``autofl.yaml`` contract, execution substrate, policy boundaries, artifacts,
+and reproducibility evidence.  The agent chooses hypotheses, edits source,
+implements algorithms, and runs candidates through existing NVFlare surfaces.
+
+``autofl.yaml`` is the human-reviewable campaign configuration, not a replacement
+for ``job.py`` or for exported NVFlare job folders.  It exposes the editable
+Auto-FL settings, fixed-budget constraints, allowed edit paths, objective,
+candidate budget, provenance, and unresolved fields.  The original ``job.py``
+remains the experiment entry point the skill and agent use to run candidates.
+
+Typical Prompt
+==============
+
+.. code-block:: text
+
+   Use the NVFlare Auto-FL skill.
+   Optimize ./job.py for validation accuracy in simulation with an
+   8-candidate budget.
+
+First Step: Deterministic Import
+================================
+
+The skill first imports the job without executing user code:
+
+.. code-block:: shell
+
+   python -m nvflare.app_common.autofl.job_importer ./job.py \
+       --metric accuracy \
+       --env sim \
+       --max-candidates 8 \
+       --output autofl.yaml
+
+The importer parses supported Recipe and FedJob patterns with Python AST
+inspection.  It extracts campaign-relevant settings into ``autofl.yaml`` and
+marks unknown or dynamic fields as unresolved instead of guessing.
+
+Trust Contract
+==============
+
+Before editing or running candidates, the skill should show the user three
+things from ``autofl.yaml``:
+
+- **Editable**: metric, environment, candidate budget, tunables, artifact
+  locations, source hash, and importer version.
+- **Unresolved**: dynamic defaults, unsupported Python semantics, missing metric
+  sources, unknown data paths, or low-confidence fields.
+- **Allowed**: files the agent may edit, fixed-budget fields it must preserve,
+  Python modules it may add under the job root, and environment or policy
+  boundaries.
+
+This makes the workflow feel native and reproducible: NVFlare owns the truth of
+the campaign settings and execution surfaces; the agent owns exploration within
+explicit constraints.
+
+Execution
+=========
+
+The bundled helper is an internal skill surface, not a public NVFlare command
+family.  It first initializes the campaign and baseline:
+
+.. code-block:: shell
+
+   python "$CODEX_HOME/skills/nvflare-autofl/scripts/run_job_campaign.py" \
+       initialize ./job.py --metric accuracy --mode max --env sim
+
+For each attempt, the agent supplies a hypothesis and receives an isolated
+candidate source directory plus ``candidate_manifest.json``:
+
+.. code-block:: shell
+
+   python "$CODEX_HOME/skills/nvflare-autofl/scripts/run_job_campaign.py" \
+       prepare ./job.py --name fedprox-variant \
+       --hypothesis "stabilize heterogeneous client updates"
+
+The agent edits that candidate source, including new Python algorithm modules
+when useful, and asks the helper to evaluate it:
+
+.. code-block:: shell
+
+   python "$CODEX_HOME/skills/nvflare-autofl/scripts/run_job_campaign.py" \
+       evaluate ./job.py --manifest <candidate_manifest.json>
+
+NVFlare computes the source diff and hash, checks allowed paths and detectable
+fixed-budget drift, executes the candidate, updates ``results.tsv`` and
+``progress.png``, and either retains the new best source or restores the prior
+best.  Built-in tunable candidates are available through the helper's
+``suggest`` action only as optional seeds; the agent remains free to implement
+new algorithms.
+
+The workflow then uses existing NVFlare execution surfaces:
+
+- Simulation jobs run through the job's configured ``SimEnv``.
+- POC and production jobs use the standard startup-kit and ``nvflare job``
+  submission, wait, download, and inspection commands.  The skill records the
+  resulting job ID, artifacts, and metric against the candidate manifest.
+- Production execution is allowed when the user requests it, but the skill must
+  not bypass normal startup-kit authentication, site policy, or job submission.
+
+Supported First Version
+=======================
+
+The first version is intentionally narrow:
+
+- Supported job surfaces: NVFlare Recipe constructors and FedJob-style scripts.
+- Supported import fields: objective metric, fixed budget fields, environment,
+  train script, allowed edit paths, and common argparse tunables.
+- Unsupported or ambiguous custom Python is preserved as unresolved review
+  fields.
+
+The default user experience should not require editing ``autofl.yaml``.  Users
+review it only when the importer reports unresolved fields or when they want to
+override the campaign configuration.

docs/user_guide/nvflare_cli/nvflare_cli.rst

@@ -47,6 +47,14 @@ Deprecated commands still exposed in help, such as ``simulator`` and
 ``authz_preview``, are documented only briefly and should not be used for new
 workflows unless you are maintaining an older setup.
 
+Agent-assisted workflows
+========================
+
+The :ref:`NVFlare Auto-FL skill <autofl_skill>` is documented with CLI
+workflows because it uses NVFlare's existing job, POC, production, and
+machine-readable output surfaces.  It is not a separate ``nvflare`` command
+group.
+
 .. toctree::
    :maxdepth: 1
 
@@ -62,5 +70,6 @@ workflows unless you are maintaining an older setup.
    cert_command
    package_command
    recipe_command
+   autofl_skill
    preflight_check
    dashboard_command

nvflare/app_common/autofl/__init__.py

@@ -0,0 +1,31 @@
+# Copyright (c) 2026, NVIDIA CORPORATION.  All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Auto-FL utilities for agent-assisted NVFlare job optimization."""
+
+__all__ = [
+    "AUTOFL_CONFIG_SCHEMA_VERSION",
+    "DeterministicJobImporter",
+    "JobImportError",
+    "dump_autofl_yaml",
+    "import_job_to_autofl_config",
+]
+
+
+def __getattr__(name):
+    if name in __all__:
+        from nvflare.app_common.autofl import job_importer
+
+        return getattr(job_importer, name)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")