Fable fleet 1: conjunction ICP basin, big-batch/GNC energy descent, ICA coarse matcher - #22
Draft
rjha18 wants to merge 20 commits into
Draft
Fable fleet 1: conjunction ICP basin, big-batch/GNC energy descent, ICA coarse matcher#22rjha18 wants to merge 20 commits into
rjha18 wants to merge 20 commits into
Conversation
… ICA coarse matcher Reframes the no-requery conjunction cell from 'identifiability barrier' to 'global optimization of a verified-aligned objective' (B1/B1b show energy's optimum is at R_true even cross-dataset). Adds: - scripts/icp_basin.py: ICP basin sweep with --emb_b_train (fills the gap that v2's 1.5-rad ICP basin was same-dataset only) - scripts/energy_descent2.py: energy descent with gradient accumulation (H-noise test) and annealed-noise graduated non-convexity (basin widening) - scripts/ica_align.py: first higher-order-statistics coarse matcher (FastICA axes + marginal-quantile Hungarian matching, restart + confidence gating); the isotropy wall is a second-order phenomenon - EXPERIMENTS_FABLE.md: pre-registered predictions + record corrections (drift_deg SVD formula is a null metric on orthogonal matrices; emb_b_train unlogged; ganconj scale confound) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…ignored) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
E2: big-batch shows the energy stall is a transient pass NEAR truth (min rank 134 at step 2000 from 1.5 rad) followed by drift to a displaced optimum — trajectory minimum, not endpoint, is the deliverable. E3: GNC refuted — smoothing destroys the fine-scale signal (the coarse landscape IS the isotropy degeneracy). E4: ICA matcher fails within-lineage control — disqualified as implemented. Adds unsupervised CSLS-criterion checkpoint selection + post-descent ICP chaining to energy_descent2.py (fleet 2: jobs 767750-4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…ship) + oracle-ceiling diagnostic Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…ogical even at the dip Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…in-vs-N, unbalanced Sinkhorn Pre-registered overnight fleet (jobs 777321-31). Consistency-graph smoke nearly solves the synthetic shifted cell (coarse 34 -> ICP 6.1) where ICA failed (2246). Encode jobs queued for 250k-point clouds (large-N gates). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…lity, 1000x curvature ratio), margin flat to N=122k, consistency-graph and unbalanced-OT dead Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…SA prototype User directives: audit implementation details (found: caches unit-normalized at encode time, norm signal never examined -> raw re-encodes running); many cheap runs permissible with principled selection; adaptive method from priors. Funnel-SA = Metropolis on subsampled full-cloud energy VALUE (no gradient path -> no direction poisoning), moves = plane rotations in the signal subspace (closed-form rank-2 updates), CSLS-mutual detector -> ICP handoff. Geodesic probe gates the funnel assumption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…obstructed); norm channel closed (architectural); SA v1 underpowered; firing the never-run cell (plain big-batch descent from 2.0/random) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…al gradient is registration-blind at long range; mechanism of the no-requery difficulty identified Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…espondence test Tests two predictions from reading arXiv:2603.21786 against our program: (1) embeddings are Gaussian except on the signal subspace (UNE x direction law -> identifiability margin = non-Gaussianity of the shared latent); (2) the most-kurtotic pursuit directions correspond across encoders under R_true (gates a theory-guided projection-pursuit matcher; also explains the GAN as adaptive projection pursuit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…subspace skeleton); pursuit directions encoder-specific — both predictions refuted, margin decomposition sharpened Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…otstrap theory: oracle precision instrumentation and dynamic CSLS tether selection Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…d knee = unsupervised percolation threshold); CSLS tether selection refuted; scale test firing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…sub-linearly with cloud size) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…weep + unsupervised signal hunt FP sweep: contaminate oracle-good tethers, measure rank vs FP rate for raw Procrustes vs trimmed vs raw+ICP (sets the precision bar). Signal hunt: score candidate tethers by csls_margin/gap12/density/cycle/nn_overlap against the oracle, report precision-lift per signal (finds what replaces the proven-blind CSLS). Smoke: trimmed Procrustes far more FP-robust than raw (0.7 FP -> 3.6 vs 83.8). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…bootstrap + unsupervised selection) for real-data test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
…ntier, next steps (resumable handoff) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Experimental fleet reframing the no-requery conjunction cell (cross-lineage x cross-dataset) from 'possible identifiability barrier' to global optimization of a verified-aligned objective — motivated by v3's own B1/B1b results (energy distance's optimum sits at R_true even cross-dataset, with a large static margin vs random rotations).
New experiments (12 Slurm jobs, 767673-767684, pre-registered in EXPERIMENTS_FABLE.md)
scripts/icp_basin.py— ICP basin sweep ON the conjunction cell (v2's 1.5-rad ICP basin was same-dataset only; the assignment-free principle predicts ICP mis-couples under support mismatch). Decides whether the coarse-matcher budget is 1.5 rad or energy's 0.5 rad.scripts/energy_descent2.py --accum 16— big-batch energy descent (H-noise vs H-landscape explanation of the 1.5-rad stall).scripts/energy_descent2.py --sigma_start 0.5— graduated non-convexity: annealed Gaussian smoothing of both clouds during SO(d) descent (assignment-free, non-adversarial analogue of the scorepot/GAN coarse-to-fine mechanism).scripts/ica_align.py— first higher-order-statistics coarse matcher (FastICA axes + marginal-quantile Hungarian matching, restarts + confidence gating). The isotropy wall that killed GW/cov-axes/spectral/QAP is a second-order phenomenon; ICA identifiability survives it.Record corrections (fold into v3.md §0)
drift_deg_from_truthinobjective_descent.pyis a null metric (SVD singular values of an orthogonal matrix are identically 1) — the ledger's 0.2-0.7° 'drift' numbers are float noise; the 'SWD moves only 0.7° in exactly the retrieval-critical subspace' narrative needs re-deriving with eigenvalue phases (fixed here).emb_b_trainwas not logged in descent result JSONs (verified the conj runs did use FineWeb via Slurm command echoes; fixed here).ganconjruns at 100k/side where the GAN is known to fail an easier cross-dataset cell that 1M solves — a negative there does not establish the barrier.🤖 Generated with Claude Code
https://claude.ai/code/session_01VGyY9VBurSmqst9R2cRhHE