[skill] evaluation: align nel-next TB2.1/SWE-bench with golden toolchain

[cjluo-nv]

What does this PR do?

Type of change: Documentation / tooling (agent skill)

Terminal-Bench 2.1 configs generated from the evaluation skill had drifted from the
canonical eval-factory config (configs/benchmarks/terminal-bench-2.1/bench.yaml). The
scoring contract already matched golden exactly — playbook, repeats: 8,
timeout_strategy: max, run_timeout: 7200, llm_kwargs.timeout: 3600, concurrency. What
had drifted was the toolchain and a few proxy-level defaults.

The drift was in the skill, not in individual configs: a config generated fresh from the
skill reproduced every stale value, so patching configs alone would not have held.

• nel-next.sh installs from the public upstream repo
  (github.com/NVIDIA-NeMo/Evaluator, default branch → 0.4.0) instead of PyPI. PyPI
  nemo-evaluator tops out at 0.3.0 and cannot reach the 0.4.x toolchain golden runs on.
  NEL_NEXT_SPEC becomes the PyPI escape hatch and now takes precedence when explicitly set;
  NEL_NEXT_ORIGIN stays overridable from .env so internal mirrors stay out of this repo.
• eval_image: document the golden pin 0.5.0.1-harbor (single source of truth:
  configs/shared/nel_next_containers.yaml) rather than 0.3.1.1-harbor as a floor.
• proxy.request_timeout 1800 → 3600 — must be >= the solver's llm_kwargs.timeout,
  else the proxy truncates long agent turns the harness is still awaiting.
• drop_params: add max_input_tokens_per_task, no_rebuild — sent by the 0.5.x harbor
  eval image; vLLM returns 400 unless stripped.
• exclude_patterns: add model_traffic.jsonl so captured request bodies stay in the run
  dir and never reach MLflow.
• http_pairs_dump interceptor (last in chain) for HTTP diagnostics.
• Sharding documented: max_concurrent/sandbox.concurrency are per shard, so
  shards: N multiplies both serving capacity and live sandboxes (N x concurrency).
• .gitignore: broaden .env / .env-* to .env* so secret backups such as
  .env.bak-tb21 cannot be staged.

This does not move the benchmark. The TB2.1 task set is pinned by a vendored registry
override that has not changed since 2026-06-03, and both 0.3.1.1-harbor and
0.5.0.1-harbor score 89 samples — so the image bump is a toolchain fix and scores stay
comparable across it.

Usage

set -a && source .env && set +a
.agents/scripts/nel-next.sh --version        # 0.4.0 (public upstream build)
.agents/scripts/nel-next.sh eval run <tb21-config>.yaml --dry-run

Testing

• nel-next.sh --version → 0.4.0, built from Evaluator.git@9dcca2ae (matches golden's
  toolchain; golden manifest.yaml evidence SHAs resolve in the public repo).
• Both TB2.1 run configs pass eval run --dry-run on that CLI with no schema errors — these
  schemas are extra="forbid", so http_pairs_dump and the new drop_params would hard-fail
  if unsupported.
• 0.5.0.1-harbor confirmed in the generated nel_eval.sbatch for both.
• pre-commit run --files <changed> — all hooks pass, no file modifications.
• Golden cross-checked against dl/JoC/competitive_evaluation/nvidia-eval-factory-benchmarking
  @ main and a reference oci-hsg run (Qwen3.6-35B-A3B TB2.1, pass@1 0.4438 over 89x8).

Not yet run: a live 2-problem canary on the new toolchain. --dry-run does not exercise
enroot credentials for the new image, the harbor agent handshake, or request-shape acceptance.

Before your PR is "Ready for review"

• Is this change backward compatible?: ✅ — NEL_NEXT_SPEC restores the previous PyPI install.
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: N/A — agent-skill docs/config; validated via --dry-run + pre-commit.
• Did you update Changelog?: N/A — no library API change.
• Did you get Claude approval on this PR?: ❌ — pending.

Additional Information

Personal run configs under .agents/skills/evaluation/runs/ are deliberately not included:
they carry internal cluster hostnames, lustre paths, account names and an AWS account id, which
do not belong in this public repo.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 039ba047-2ad9-4265-9a8c-7cfd3abead57

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.59%. Comparing base (14b20c0) to head (821ed92).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2063      +/-   ##
==========================================
- Coverage   67.00%   66.59%   -0.42%     
==========================================
  Files         520      521       +1     
  Lines       59545    60621    +1076     
==========================================
+ Hits        39900    40371     +471     
- Misses      19645    20250     +605     

Flag	Coverage Δ
unit	55.39% <ø> (+0.14%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.