Repository navigation
feat: production FQL actor, critic and offline transition training - #293
felixmaximilian wants to merge 8 commits into
Conversation
|
Independent source-contract review at def7325: no remaining blocker found for the explicitly opt-in single-transition/state FQL scope. Compared gradient routing and old-critic EMA to pinned actual upstream. Findings fixed: NaN/unstable episode IDs bypassing disjointness; coercible boolean config; overflowing physical action spans; nonidentity tiny-action normalization; complex input silently cast to real. Independent26CPUtests pass. Existing dataset adapter deliberately requires explicit rewards/masks/next states and does not infer human failure labels; exact episode transition/timeout semantics remain a benchmark-adapter gate. Generic shuffled Lightning loader is not upstream with-replacement training. This is source-contract approval ONLY: strict chained numerical oracle currently retains a diagnosed near-zero-gradient Adam failure, and transferred/native8-seed G2 quality gates remain pending. No G2-complete claim, merge authorization or D12 training/deployment approval. |
|
Incremental independent source review now covers0cbc3f46f373110fd785398573b9a6a8ca2e76d8. The actual clipping derivative discrepancy at exact±1 is corrected in actor-Q loss (matching JAX half tie), and physical endpoint roundoff is clamped only after validating true physical bounds. Independently28CPUtests pass. This supersedes previous commit-scoped source review only; amended full numerical oracle and transferred/native8-seed quality remain pending. No merge/deployment or G2-complete claim. |
|
Independent incremental source-contract review: approved engineering scope at d91597c. Shared Adam factory retains single_tensor default, exposes optional foreach, rejects fused. Requested and verified checkpoint resume guards bind constructor execution/LR to saved contract and actual optimizer param-group settings (with explicit legacy single-tensor compatibility). Independently ran tests/test_fql.py: 35 passed. Numerical/closed-loop validation for the chosen native runtime/mode remains a separate G2 gate; this is not D12 training or deployment approval. |
|
Independent final incremental engineering review: exact e608fac approved in scoped source review; 47 CPU tests independently pass. Shared gradient validator preserves every existing finite-gradient predicate, including any present target-critic gradients, and runs before EMA/Adam; only per-device host synchronization is aggregated. No training math or safety guard was removed. Closed-loop transferred-checkpoint G2-B separately passes (95.4167% vs95.9167%, paired95% CI difference [-1.25,+0.0833]pp inside frozen±2pp); native eight-seed G2-C remains pending, so no D12 training/deployment approval. |
Scope
Opt-in production FQL implementation on
main-based branch: official low-dimensional architecture with a flow behavior prior, a separate one-step actor, twin critics and target critics. It adds explicit-noise losses, isolated gradients, pre-optimizer target EMA, Lightning training/checkpoint contracts, normalization, and an explicitly labeled transition datamodule. FM/VQ/RTC defaults are unchanged. This is H1 state-based FQL, not an image/H50/RTC/D12 serving policy.Validation
Evidence and scope limits
Benchmark-side receipts and frozen reports are in yaak-ai/rmind-bench#7; the independent audit is in yaak-ai/rmind-bench#4. The benchmark imports the actual production source rather than a duplicate objective. Task-unaware OGBench play is not labeled failed truck experience. Image conditioning, action chunks, RTC, and D12 rollouts require a separate architecture/data/closed-loop gate and PR. No D12 training, deployment or actuation is authorized by this PR.