Skip to content

feat(engine): validate Dream Pathway against MindSpan longitudinal dataset - #918

Open
forgespectrayan wants to merge 2 commits into
mainfrom
feature/dream-pathway-mindspan-validation
Open

forgespectrayan wants to merge 2 commits into
mainfrom
feature/dream-pathway-mindspan-validation

Conversation

@forgespectrayan

@forgespectrayan forgespectrayan commented Sep 14, 2026

Copy link
Copy Markdown
Member

Summary

Empirically validates Spector's 7th canonical cognitive pathway (Dream Pathway) against the full 20-year longitudinal MindSpan v2-memory dataset (360MB, 18,404 semantic records, 27,664 index entries, 61,474 Hebbian edges) using fast APFS copy-on-write cloning, and wires LlmProvider to SceneConstructRelay for generative creative synthesis validated with local Ollama (llama3.1:latest).

Key Changes

  1. LlmProvider Creative Dream Synthesis:
    • Injected LlmProvider through SpectorMemoryFactory, DreamPathway, and DreamSignal into SceneConstructRelay.
    • Replaced raw template slot concatenations (sem-0-XXXXX) with rich, evocative 1-2 sentence dream narratives and emergent cross-domain insight hypotheses synthesized by LLM.
    • Preserved stochastic temperature modulation by embedding synthesized narratives into vector space and injecting Hartmann-scaled Gaussian regularizing noise.
    • Built resilient fallback (ErrorPolicy.DEGRADE_GRACEFULLY) to deterministic structural templates if LLM is unavailable or offline.
  2. Durable Write-Back Wiring: Injected RememberPathway into DreamPathway and DreamSignal. Qualifying surviving dream scenes ($Q \ge \theta_{\text{persistence}}$) are committed to durable semantic memory with FLAG_DREAMED (0x80) | FLAG_SIMULATED (0x20) and source DREAMED / THOUGHT_EXPERIMENT.
  3. Lexical Fusion Candidate Gating: Refined RecallCandidateGatherer to preserve consolidationFlags from slab headers and exclude simulated/dreamed candidates during lexical/BM25 fusion when allowSimulated == false.
  4. Dedicated Benchmark Runner & Local Ollama Validation:
    • Implemented MindSpanDreamRunner and MindSpanDreamBenchmarkTest in spector-bench with automatic OllamaLlmProvider detection.
    • Validated autonomous Targeted Memory Reactivation (TMR) seed sampling across longitudinal slabs.
    • Tested 12-relay conduction across 3 canonical modes: REM ($T=2.0$), THOUGHT_EXPERIMENT ($T=0.5$), DAYDREAM ($T=1.0$).
    • Validated active Hebbian LTD downscaling ($\Delta w < 0$) on noise/incoherent combinations.
    • Verified 100.00% confabulation immunity on factual queries (allowSimulated = false, 0 confabulations detected across 20 years of memories).
    • Verified speculative hypothesis retrieval on exploratory queries (allowSimulated = true, retrieving persisted rich dreamed insights).

Verification

  • MindSpanDreamBenchmarkTest with local Ollama (llama3.1:latest): PASSED (seeds=8, scenes=16, factualConfabulations=0, exploratoryRetrieved=9)
  • spector-memory dream unit tests: 11/11 tests PASSED
  • MindSpanAismeRunnerTest: PASSED (zero regression on AISME recall)
  • PostBootstrapSyspropBanTest: PASSED (100% compliant with ADR-0031)
  • Zero changes to spector-kernel

Co-authored-by: Bharat Joshi bharatjoshi@spectrayan.com

…mory

- Wire RememberPathway into DreamPathway and DreamSignal for durable write-back
- Persist qualifying dream scenes with FLAG_DREAMED and MemorySource.DREAMED
- Fix RecallCandidateGatherer consolidationFlags propagation and allowSimulated gating
- Add MindSpanDreamRunner and MindSpanDreamBenchmarkTest verifying TMR and zero confabulation
- Assert 100% confabulation immunity on factual recall and positive exploratory retrieval

Co-authored-by: Bharat Joshi <bharatjoshi@spectrayan.com>
@forgespectrayan
forgespectrayan requested a review from a team as a code owner September 14, 2026 03:43

// Fast path: attempt macOS APFS copy-on-write clone (takes ~13ms for 360MB)
try {
Process p = new ProcessBuilder("cp", "-c", "-R", sourceDir.toString(), targetDir.toString())
…eam synthesis

- Wire LlmProvider through SpectorMemoryFactory, DreamPathway, and DreamSignal to SceneConstructRelay
- Enable LLM-driven generation of vivid dream narrative scenes and emergent cross-domain hypotheses
- Embed synthesized narratives into vector space under Hartmann-scaled stochastic noise
- Support graceful fallback to structural templates when LLM is unavailable or times out
- Validate locally with Ollama (llama3.1:latest) against the MindSpan v2-memory benchmark

Co-authored-by: Bharat Joshi <bharatjoshi@spectrayan.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants