diff --git a/.gitignore b/.gitignore index dbe68211..f6d92366 100644 --- a/.gitignore +++ b/.gitignore @@ -3,8 +3,6 @@ .coveragerc .notes/* !.notes/.gitkeep -!.notes/eeglab-migration-gap-audit.md -!.notes/implementation-notes.html # Python # Byte-compiled / optimized / DLL files diff --git a/.notes/eeglab-final-epic-closeout.md b/.notes/eeglab-final-epic-closeout.md deleted file mode 100644 index b2b2c353..00000000 --- a/.notes/eeglab-final-epic-closeout.md +++ /dev/null @@ -1,180 +0,0 @@ -# EEGPrep Final Standalone Epic Closeout Evidence - -Date: 2026-06-07 -Phase issue: #165 -Epic issue: #157 -Branch: `phase/165-final-integration-release-hardening` -Base: `origin/feature/eeglab-full-standalone-completion` -Final epic PR target after this phase: `origin/develop` - -Phase 8 closes the phase stack on the epic branch. It prepares evidence for the -future epic PR to `develop`, but does not open that final PR. - -## Phase Issues and PRs - -| Phase | Issue | PR | Scope | Result | -| --- | --- | --- | --- | --- | -| 1 | #158 | #166 | Final standalone audit matrix, validator, runtime contract, docs architecture | Merged | -| 2 | #159 | #171 | clean_rawdata and FIRFilt bundled-plugin completion | Merged | -| 3 | #160 | #169 | DIPFIT and source-localization standalone parity | Merged | -| 4 | #161 | #170 | STUDY PAC, LIMO-compatible design, and advanced statistics boundaries | Merged | -| 5 | #162 | #168 | Large-dataset `.fdt`, memmap, and `storedisk` semantics | Merged | -| 6 | #163 | #167 | ICLabel, viewprops, and component diagnostic parity | Merged | -| 7 | #164 | #172 | Standalone Sphinx manual, tutorials, API pages, and help resources | Merged | -| 8 | #165 | This branch | Integration, QA, release hardening, and evidence | In review | - -## Feature Summary - -The merged epic branch now covers the remaining standalone EEGLAB parity areas -identified after the core parity PR: - -- A machine-readable final parity matrix and validator for bundled plugins, - object/storage semantics, optional-toolbox boundaries, and documentation. -- clean_rawdata standard ASR, calibration-time Riemannian ASR support, explicit - full-Riemannian limitation, artifact diagnostics, and FIRFilt helper/dialog - parity. -- DIPFIT standalone spherical settings, grid search, nonlinear fitting, - multifit, leadfield, dipplot, coordinate transforms, and explicit source - backend limits. -- STUDY PAC compute/cache/read/plot workflows, LIMO-compatible design-matrix - export, neighbor/interpolation helpers, and explicit source-statistics - boundaries. -- Python-native large-dataset save/load, `.fdt` sidecars, memory maps, - `storedisk` offload/retrieve behavior, and GUI/console session synchronization. -- ICLabel default-network runtime behavior, label statistics, viewprops-style - diagnostics, and explicit alternate-network boundaries. -- EEGPrep-owned Sphinx user manual, migration notes, GUI plus console tutorials, - BIDS workflow docs, bundled plugin docs, generated API pages, and packaged - help resources. - -## Matrix Closeout - -`docs/parity/eeglab_final_parity_matrix.json` now validates against the full -final-epic reference surface: - -- 31 grouped rows cover 180 final-epic EEGLAB reference paths. -- Status counts: 21 `implemented`, 3 `consolidated`, 2 `optional_dependency`, - 2 `partial`, 2 `stale_skip`, and 1 `matlab_runtime_skip`. -- There are no remaining `port` or `docs_gap` rows. -- The two `partial` rows are intentional backend boundaries, not forgotten work: - DIPFIT MRI/BEM/LORETA/FieldTrip source workflows and STUDY-level - FieldTrip/source-statistics workflows. - -Phase 8 reclassified the six Phase 7 documentation rows from `docs_gap` to -`implemented` after verifying that the completed Sphinx manual covers console -history migration, event/indexing tutorials, STUDY workflows, source/DIPFIT -boundaries, time-frequency/visual workflow notes, and BIDS tutorials. - -## Phase 8 Findings Fixed - -- The final matrix still had six Phase 7 documentation rows marked `docs_gap` - after PR #172 merged. Phase 8 reclassified those rows to `implemented` and - added regression coverage that no final closeout row remains `port` or - `docs_gap`. -- The MATLAB-enabled parity run exposed an inconsistent synthetic HDF5 fixture: - `xmin=-1`, `pnts=1000`, `srate=500`, and `xmax=1.0`. EEGLAB/EEGPrep timing - semantics make the final point `0.998`, so Phase 8 corrected the fixture and - assertion in `tests/test_pop_loadset_h5.py`. -- OC autoreview found that the time-frequency/movie documentation row needed - explicit user-guide coverage. Phase 8 added supported time-frequency and ERP - image wrapper guidance to `docs/source/user_guide/preprocessing_pipeline.rst` - and the EEG movie boundary to `docs/source/user_guide/visual_parity.rst`. -- The first MATLAB parity attempt found a truncated ignored - `sample_data/EmotionValence.set` download. Phase 8 refreshed that local - fixture to the S3-reported 203,941,008 bytes before rerunning the suite. - -## Runtime Independence - -Runtime package code must not read, import from, or shell out to -`src/eegprep/eeglab`. The vendored tree remains a development and parity-test -oracle only. - -Phase 8 verified this with `tests/test_runtime_eeglab_independence.py` and a -source scan for direct `src/eegprep/eeglab`, `eegprep.eeglab`, importlib -resource, or package-root path-join dependencies. Development validators and -visual parity tools may read the vendored reference tree. - -## Public API, Help, Package Data, and Menu Inventory - -Phase 8 kept the public API/menu/help/package evidence in tested surfaces: - -- `tests/test_package_exports.py` covers lazy public exports. -- `tests/test_public_api_examples.py` covers documented API examples and - package-data declarations. -- `tests/test_guifunc_pophelp_chansel.py` covers packaged Markdown help - resources, `pophelp`, and implemented menu help targets. -- `tests/test_menu_placeholder_inventory.py` covers menu placeholder metadata. -- `tests/test_gui_main_window.py` covers main-window menu/help/session wiring. - -## Visual Parity Attachment Inventory - -Visual artifacts are attached to phase PR comments rather than committed: - -- PR #167, ICLabel/viewprops: `iclabel_pop_prop_extended_dashboard`, - `pop_icflag_dialog`, `pop_iclabel_dialog`, and `pop_viewprops_dialog` - side-by-side images. -- PR #169, DIPFIT: `pop_dipfit_settings`, `pop_dipfit_gridsearch`, - `pop_dipfit_nonlinear`, `pop_dipplot`, `pop_multifit`, `pop_leadfield`, - `pop_dipfit_loreta`, and `pop_dipfit_headmodel` evidence. -- PR #171, FIRFilt: `pop_kaiserbeta_dialog`, `pop_firwsord_dialog`, - `pop_firpmord_dialog`, `pop_xfirws_dialog`, and refreshed - `pop_firpmord_dialog_review_fix` evidence. -- PRs #166, #168, #170, and #172 did not add new GUI dialog layouts requiring - fresh visual attachments. - -## GUI Agent Mixed Workflow QA - -Computer Use inspected the live `eegprep-gui --window-menu-bar` window and -confirmed the visible startup state: File and Help enabled, dataset-dependent -Edit/Tools/Plot/Study/Datasets menus disabled, and the EEGLAB-style startup -instructions visible. Computer Use click actions against the Python-hosted -Qt app were rejected by the tool as inactive immediately after state capture, -so Phase 8 continued the requested flow QA with Qt-driven actions against the -real main window. - -The mixed workflow QA script exercised startup menus, Help menu opening, -packaged `pophelp` dispatch, dataset storage and GUI refresh, GUI dataset -retrieve followed by console namespace inspection, and a bare console -`pop_reref(EEG, [])` call followed by GUI refresh. - -## Accepted Non-Goals - -These are explicit final-epic boundaries rather than Phase 8 omissions: - -- Full Manopt-backed Riemannian ASR processing remains an optional dependency - decision; standard ASR and calibration-time Riemannian behavior are supported. -- DIPFIT MRI-derived BEM headmodel creation, AFNI atlas clipping, LORETA source - analysis, and FieldTrip source-statistics workflows remain explicit backend - limits. -- Full LIMO model fitting, result computation, and browsing remain external - backend workflows; EEGPrep owns design-matrix preparation. -- ICLabel `lite` and `beta` network artifacts remain explicit MATLAB/Octave - engine paths until EEGPrep packages tested standalone assets. -- MATLAB object overloads, developer tests, command-window shims, and third-party - plugin ecosystems remain consolidated, skipped, or extension-owned rather than - one-for-one runtime ports. - -## Verification Log - -Phase 8 verification is run from this branch. Results are updated before the -phase PR is opened. - -| Command | Result | -| --- | --- | -| `uv sync --group dev --extra gui --extra console --extra docs --extra torch` | Passed | -| `uv pip install /Applications/MATLAB_R2026a.app/extern/engines/python` | Passed: installed local `matlabengine==26.1` for MATLAB parity verification | -| `uv run --no-sync python -m tools.eeglab_final_parity_matrix --json` | Passed: `ok: true`, 31 rows, 180 expected paths | -| `uv run --no-sync pytest tests/test_eeglab_final_parity_matrix.py tests/test_pop_loadset_h5.py::TestPopLoadsetH5::test_basic_h5_loading --tb=short` | Passed: 13 passed | -| `uv run --no-sync pytest tests/test_runtime_eeglab_independence.py tests/test_guifunc_pophelp_chansel.py tests/test_public_api_examples.py tests/test_package_exports.py tests/test_menu_placeholder_inventory.py` | Passed: 37 passed | -| `uv run --no-sync pytest tests/test_gui_main_window.py` | Passed: 69 passed | -| `uv run --no-sync pytest tests/test_console_workspace.py` | Passed: 86 passed | -| `uv run --no-sync ruff check .` | Passed | -| `uv run --no-sync ruff format --check .` | Passed | -| `uv run --no-sync ty check` | Passed | -| `uv run --no-sync sphinx-build -b html docs/source docs/_build/html` | Passed | -| `uv run --no-sync pytest tests/test_visual_parity.py` | Passed: 26 passed | -| `EEGPREP_SKIP_MATLAB=1 uv run --no-sync pytest -m "not slow" --tb=short` | Passed: 1873 passed, 209 skipped, 12 deselected | -| `uv run --no-sync pytest -m "matlab or octave" --tb=short` | Passed: 342 passed, 21 skipped, 1818 deselected | -| GUI Agent mixed workflow QA script | Passed: startup/menu/help, GUI retrieve to console sync, console bare `pop_reref` to GUI refresh | -| `.agents/skills/oc-autoreview-adapted/scripts/autoreview --mode local --codex-bin /tmp/codex-fast-autoreview` | Passed clean after fixing accepted docs findings | -| `./pre-commit.py --fix` | Passed on the staged Phase 8 files | diff --git a/.notes/eeglab-final-parity-audit.md b/.notes/eeglab-final-parity-audit.md deleted file mode 100644 index b0635c77..00000000 --- a/.notes/eeglab-final-parity-audit.md +++ /dev/null @@ -1,168 +0,0 @@ -# EEGPrep Final Standalone Parity Audit - -Audit date: 2026-06-07 -Epic: #157 -Phase issue: #158 -Machine-readable contract: `docs/parity/eeglab_final_parity_matrix.json` -Validator: `uv run --no-sync python -m tools.eeglab_final_parity_matrix --json` - -## Purpose - -This audit expands the completed core parity matrix into the final standalone -product-completion surface for EEGPrep. It covers the remaining non-stale -workflows that are not fully represented by -`docs/parity/eeglab_core_parity_matrix.json`: - -- bundled plugin depth for `clean_rawdata`, `firfilt`, `ICLabel`/viewprops, and - `dipfit`; -- MATLAB object/storage folders `@eegobj`, `@memmapdata`, and `@mmo`; -- optional-toolbox workflows such as LIMO, PAC, FieldTrip-style STUDY helpers, - Riemannian ASR, and DIPFIT fitting; -- EEGLAB tutorial/doc surfaces that should become EEGPrep-owned Sphinx docs. - -This phase does not port feature behavior. It defines the rows, status values, -phase ownership, optional-dependency rules, documentation architecture, and -validator behavior that later phase agents must use. - -## Status Taxonomy - -Rows in the final matrix use these statuses: - -- `implemented`: EEGPrep has the standalone behavior; the responsible phase - owns verification, docs, and final evidence. -- `partial`: EEGPrep has part of the behavior, but important options, GUI - paths, diagnostics, numerical parity, or docs still need phase work. -- `port`: a useful workflow is not implemented yet and should be ported or - redesigned as native Python/Qt behavior. -- `consolidated`: EEGPrep intentionally covers the behavior through a different - Python helper or public contract instead of a same-name file. -- `stale_skip`: a MATLAB demo, test, packaging helper, or obsolete alias is not - a user workflow. These rows require the full stale-policy object with every - field set to `false`. -- `matlab_runtime_skip`: MATLAB path, command-window, GUI shim, or toolbox - activation behavior that must not exist in installed EEGPrep runtime code. -- `optional_dependency`: a real scientific workflow needs a substantial backend - decision. The row must name the dependency, fallback behavior, user-facing - message, and phase contract. -- `external_plugin`: behavior belongs to an external plugin ecosystem and - should use EEGPrep extension contracts rather than core package code. -- `docs_gap`: a documentation/tutorial surface that Phase 7 must write after - feature phases define final behavior. - -All non-skip rows must name a responsible phase and the matching phase issue. -Skip rows must have `responsible_phase: "none"` and `phase_issue: null`. - -## Phase Ownership - -- Phase 2 / #159 owns `clean_rawdata` and FIRFilt rows, including standard ASR, - Riemannian ASR optional-backend decisions, `vis_artifacts`, FIR order - calculators, boundary helpers, reports, and frequency-response plotting. -- Phase 3 / #160 owns DIPFIT settings, fitting, FieldTrip/source-localization - boundaries, atlas and coordinate transforms, leadfield/LORETA workflows, and - dipole plotting evidence. -- Phase 4 / #161 owns optional LIMO/PAC/STUDY statistics behavior and - FieldTrip-style STUDY neighbor/interpolation/source workflows. It must - coordinate with Phase 3 for source-localization assumptions. -- Phase 5 / #162 owns Python-native large-dataset storage, `storedisk`, - `option_memmapdata`, and the product decision for `@memmapdata`/`@mmo` - semantics. It should not port MATLAB overloads one-for-one. -- Phase 6 / #163 owns ICLabel, label statistics, viewprops, component-property - diagnostics, alternate runtime/network decisions, and visual evidence. -- Phase 7 / #164 owns the EEGLAB-style Sphinx docs architecture and all - `docs_gap` tutorial rows. It should merge after feature phases describe - actual completed behavior rather than intentions. -- Phase 8 / #165 owns final integration, release hardening, docs build, - non-slow tests, MATLAB parity where available, visual parity suite, GUI Agent - mixed-flow QA, and evidence rollup. - -## Phase 2 Completion Update - -Phase 2 / #159 moved the useful `clean_rawdata` and FIRFilt rows from intent to -verified standalone behavior. Standard clean_rawdata/ASR remains implemented as -native Python, `vis_artifacts` diagnostics are packaged and wired through -`pop_clean_rawdata`, and FIRFilt now exposes EEGLAB-named helper functions, -order dialogs, boundary-aware filtering helpers, reports, and frequency-response -plotting without any runtime dependency on the vendored EEGLAB tree. - -Full Riemannian ASR processing remains an `optional_dependency` row by design. -EEGPrep supports the calibration-time Riemannian estimate through -`useriemannian='calib'`, but it does not vendor Manopt or fake full -Riemannian processing. Explicit full-processing requests must fail with a clear -limitation until a tested Python manifold-optimization backend is selected. - -## Optional-Dependency Rules - -EEGPrep should prefer standalone Python behavior for core preprocessing, data -structures, GUI/console state, saved-file behavior, and bundled plugin workflows -that can be tested without external scientific toolboxes. - -Use `optional_dependency` only when the workflow requires a substantial backend -that can be installed, versioned, tested, and documented. These rows must not -produce fake outputs. Until a backend is selected, user-facing functions should -raise clear limitations that name the missing backend and point to EEGPrep docs. - -Use `external_plugin` for broad third-party ecosystems that do not belong in -core EEGPrep. Those workflows should be built through the extension API, -catalog/trust model, packaged help, tests, and docs. - -## Documentation Architecture - -Phase 7 should reorganize and expand Sphinx docs into an EEGLAB-style manual -with EEGPrep-specific behavior: - -1. Installation and optional dependencies. -2. Concepts guide: EEG structures, events, epochs, channel locations, ICA, - STUDY, history, `EEG`, `ALLEEG`, `CURRENTSET`, and indexing boundaries. -3. GUI tutorials and menu workflows. -4. Command line, `eegprep-console`, `LASTCOM`, `ALLCOM`, and script replay. -5. Preprocessing, filtering, cleaning, and artifact diagnostics. -6. ICA, ICLabel, rejection, and visual diagnostics. -7. STUDY, statistics, PAC, source localization, and optional backend limits. -8. Bundled plugins and external extensions. -9. API reference. -10. EEGLAB migration notes mapping familiar MATLAB workflows to EEGPrep GUI, - Python, and console workflows. -11. Developer parity contracts, matrix maintenance, visual evidence, and the - standalone runtime boundary. - -The current docs already have useful pages under `docs/source/user_guide/`, -`docs/source/api/`, and `docs/source/examples/`. Phase 7 should reorganize and -extend those pages rather than generating API dumps or copying EEGLAB prose. - -## Runtime Contract - -The installed `eegprep` package must not read, import from, or shell out to -`src/eegprep/eeglab`. That checkout is a development oracle for audits, parity -tests, and tooling only. Runtime help text, options, sample resources, and docs -must be EEGPrep-owned packaged resources. - -This phase adds `tests/test_runtime_eeglab_independence.py` to scan package -Python files for vendored-reference dependency patterns. Tooling under `tools/` -may read the vendored EEGLAB tree because validation is a development task. - -## Validator Contract - -`tools.eeglab_final_parity_matrix` discovers 180 final-epic EEGLAB reference -paths from the vendored checkout: - -- plugin paths from the four bundled plugin roots, excluding MatConvNet and - Manopt third-party library internals/examples/tests; -- object/storage MATLAB class folder files; -- EEGLAB tutorial scripts and Live Scripts. - -Rows may group several source paths into one workflow, but each discovered -reference path must appear exactly once. Missing paths, duplicated paths, -missing optional-dependency contracts, stale-skip policy mistakes, missing docs -architecture sections, and inconsistent phase ownership fail validation. - -The existing core command remains separate and must still pass: - -```bash -uv run --no-sync python -m tools.eeglab_parity_matrix --json -``` - -The final epic command is: - -```bash -uv run --no-sync python -m tools.eeglab_final_parity_matrix --json -``` diff --git a/.notes/eeglab-migration-gap-audit.md b/.notes/eeglab-migration-gap-audit.md deleted file mode 100644 index 4599399a..00000000 --- a/.notes/eeglab-migration-gap-audit.md +++ /dev/null @@ -1,241 +0,0 @@ -# EEGPrep Remaining EEGLAB Migration Gap Audit - -Audit date: 2026-06-07 -Branch audited: `feature/eeglab-core-parity-completion` / PR #152 -Primary reference: `docs/parity/eeglab_core_parity_matrix.json` - -## Summary - -This file is the current committed audit of what still remains after the -remaining-core-parity epic. It supersedes the earlier pre-epic audit that listed -items 1-7 as open work. - -PR #152 completes or explicitly classifies the first seven migration categories: - -1. Long-tail helper coverage -2. Missing or legacy `pop_*` entry points -3. Unsupported options in implemented user-facing functions -4. STUDY and group-level depth -5. Statistics package -6. Time-frequency internals -7. File-format and channel-location long tail - -The machine-readable matrix now covers all in-scope EEGLAB functions for those -categories: - -- `uv run --no-sync python -m tools.eeglab_parity_matrix --json` -- 608 rows cover 608 expected in-scope EEGLAB functions. -- Status counts: 238 `implemented`, 99 `consolidated`, 104 `stale_skip`, - 121 `matlab_runtime_skip`, and 46 `external_dependency_skip`. -- There are no remaining `port`, `partial`, or unclassified rows for the - categories covered by PR #152. - -## Completed In PR #152 - -The epic converted the original gap audit into an enforceable parity matrix and -then closed the useful core gaps through phase PRs: - -- Phase 1: parity matrix, staleness triage, and standalone runtime contracts. -- Phase 2: file-format and channel-location long-tail helpers. -- Phase 3: EEGLAB-style statistics helpers. -- Phase 4: time-frequency internals and legacy time-frequency entry points. -- Phase 5: unsupported options in already user-facing functions. -- Phase 6: deeper STUDY/group-level helpers. -- Phase 7: remaining audit-approved helpers and `pop_*` wrappers. -- Phase 8: closeout docs, help resources, QA, and matrix validation. - -The follow-up issues for time-warped `newtimef`, exact `correct_mc` random-symbol -distribution fitting, PAC classification, and STUDY long-tail helper depth are -also resolved in the same PR stack. - -## What Still Remains - -The remaining items are no longer generic “missing same-name files.” They are -explicitly outside the current core parity scope, intentionally skipped because -they are stale/MATLAB-only, or dependent on external runtimes/toolboxes. Future -work should start from these product areas, not from a blind same-name port. - -### 1. MATLAB Runtime And Figure-Helper Skips - -Many EEGLAB files are MATLAB command-window, path, deployed-app, figure-editing, -or low-level GUI compatibility helpers. EEGPrep should not port these -one-for-one unless a real EEGPrep user workflow needs them. - -Examples include: - -- `abouteeglab`, `eeg_cache`, `eeg_eval`, `eeg_global`, `eeglab_execmenu`, - `eeglab_new`, and `eeglab_options` -- EEGLAB help menu wrappers such as `eeg_helpadmin`, `eeg_helpgui`, - `eeg_helppop`, `eeg_helpstudy`, and `eeg_helptimefreq` -- MATLAB dialog or path wrappers such as `questdlg2`, `warndlg2`, - `uigetfile2`, `uiputfile2`, and `removepath` -- MATLAB plotting utilities such as `axcopy`, `copyaxis`, `plotcurve`, - `plotdata`, `ploterp`, `plotmesh`, `plotsphere`, `textsc`, and `sbplot` - -Recommended handling: keep these rows classified in the parity matrix. Port only -when a concrete EEGPrep GUI/API path requires the behavior, and implement it as -native Python/Qt behavior rather than a MATLAB-runtime imitation. - -### 2. External Dependency And Toolbox-Backed Workflows - -Some EEGLAB functions depend on external toolboxes, MATLAB-specific runtimes, or -large plugin ecosystems. These are intentionally not silently faked in EEGPrep. - -Examples include: - -- LIMO workflows: `pop_limo`, `pop_limoresults`, `std_limo`, - `std_limodesign`, `std_limoresults`, and `std_readfilelimo` -- FieldTrip/neighbour/DIPFIT-dependent STUDY helpers such as - `std_prepare_neighbors`, `std_interp`, `std_dipplot`, and - `std_dipoleclusters` -- Legacy ICA backends such as `binica`, `jader`, `sobi`, `acsobiro`, - `fastif`, and old `runica_ml*` variants -- Direct legacy BIOSIG/EGI import wrappers where a supported modern import path - already exists or an external backend is required -- PAC compute/plot/cache helpers such as `pac`, `pac_cont`, `std_pac`, - `std_pacplot`, and `std_readpac` - -Recommended handling: create separate product epics only when EEGPrep can offer -a tested standalone implementation or a clearly documented optional dependency. -Do not add placeholder math or fake cache files just to match names. - -### 3. MATLAB Object And Memory-Mapped Infrastructure - -EEGLAB has MATLAB class-style folders that are not one-for-one Python concepts: - -- `functions/@eegobj` -- `functions/@memmapdata` -- `functions/@mmo` - -This matters for full MATLAB `storedisk` and memory-mapped dataset behavior. -EEGPrep currently uses explicit Python `EEGPrepSession`, `ALLEEG`, and dataset -storage semantics instead. - -Recommended handling: treat this as a future storage/performance design epic, -not a MATLAB class port. If EEGPrep needs large-dataset lazy loading, design a -Python-native data backend with tests for GUI/console synchronization, -`pop_newset`, save/load behavior, and STUDY workflows. - -### 4. Bundled Plugin Depth - -The bundled in-repo plugin surfaces are represented, but not all plugin internals -are complete one-for-one ports. - -Remaining plugin-depth areas: - -- `clean_rawdata`: exact Riemannian ASR processing parity, full - `vis_artifacts` behavior, and Manopt-backed MATLAB helper depth. -- `firfilt`: lower-level helper coverage such as detailed reports, inverse - order helpers, minimum-phase helpers, frequency-response plotting, and order - calculator dialogs. -- `ICLabel` / viewprops: Phase #163 closes the standalone default-network, - `eeg_icalabelstat`, and native Qt/Python diagnostic-browser surface. The - EEGLAB `lite`/`beta` network artifacts remain explicit MATLAB/Octave - passthrough choices rather than silently emulated standalone assets. -- `DIPFIT`: lower-level grid/nonlinear/reject/dipplot helpers, manual/batch - dialogs, atlas conversion helpers, and private transform utilities. - -Recommended handling: split these into plugin-specific epics. Each plugin should -define what “standalone EEGPrep parity” means, which external assets are allowed, -and which EEGLAB MATLAB internals should stay unported. - -### 5. External Plugin Ecosystem - -EEGPrep now has extension infrastructure, but the broad EEGLAB external plugin -ecosystem is not part of core EEGPrep. - -Examples: - -- ERPLAB -- LIMO as a full external statistics workflow -- SIFT -- NFT -- MFF/importer plugin ecosystems -- Lab-specific processing plugins - -Recommended handling: external plugin work should use EEGPrep’s extension -contracts, catalog/trust model, documentation, and extension-development skill. -Do not merge third-party plugin behavior into core EEGPrep unless it becomes a -maintained bundled plugin with tests, docs, packaging, and GUI/console support. - -### 6. Docs, Tutorials, And User Education - -EEGPrep has EEGPrep-owned help resources and user-facing docs for the new epic -work, but EEGLAB’s full tutorial corpus is larger than the ported docs. - -Remaining useful docs work: - -- End-to-end tutorials comparable to EEGLAB’s practical workflows. -- More task-oriented examples for STUDY, EEGBrowser, extension authoring, file - I/O, time-frequency, and statistics. -- User-facing migration notes for EEGLAB users moving MATLAB commands to - EEGPrep Python/console workflows. -- Curated visual parity evidence index for major GUI surfaces. - -Recommended handling: keep help Markdown next to user-facing `pop_*` features, -but plan tutorials as product documentation rather than generated API dumps. - -### 7. Ongoing Parity Matrix Maintenance - -The parity matrix is now the source of truth for the first seven audit -categories. Future feature work should update it whenever an EEGLAB-facing -function is added, consolidated, or intentionally skipped. - -Required behavior: - -- `tools/eeglab_parity_matrix.py` must stay green. -- Runtime package code must not depend on `src/eegprep/eeglab`. -- New GUI features need visual parity evidence. -- New console/API features need replayable history and `eegprep-console` - synchronization tests where relevant. -- New numerical behavior should have MATLAB parity tests when deterministic and - feasible. - -## Not Recommended - -Avoid these patterns in future migration work: - -- Blind one-file-to-one-file ports of stale MATLAB helpers. -- Fake implementations that return plausible shapes but do not perform the - EEGLAB workflow. -- Runtime fallbacks that read the vendored EEGLAB checkout. -- Adding external-toolbox behavior without an explicit optional dependency, - install docs, tests, and clear user-facing failure mode. -- Implementing old MATLAB GUI shims when the right EEGPrep answer is a native - Qt or Python API surface. - -## Next Planning Step - -Epic #157 now turns the remaining product areas above into a scoped issue tree. -Its Phase 1 contract lives in: - -- `.notes/eeglab-final-parity-audit.md` -- `docs/parity/eeglab_final_parity_matrix.json` -- `tools/eeglab_final_parity_matrix.py` - -The final matrix assigns concrete phase ownership for bundled plugin depth, -object/storage semantics, optional-toolbox workflows, and docs/tutorial gaps. -Future phase agents should update that matrix instead of reclassifying product -scope from this prose audit. - -Before the final epic started, the strongest candidates were: - -1. Bundled plugin depth, split by plugin family. -2. Large-dataset storage and memory mapping semantics. -3. External dependency workflows such as LIMO or advanced DIPFIT. -4. User documentation/tutorial parity for completed core workflows. - -Before changing core parity rows, run: - -```bash -uv run --no-sync python -m tools.eeglab_parity_matrix --json -``` - -Before changing final epic rows, run: - -```bash -uv run --no-sync python -m tools.eeglab_final_parity_matrix --json -``` - -Then decide whether the work changes the existing matrix rows or belongs to a -new matrix/category outside PR #152’s original scope. diff --git a/.notes/implementation-notes.html b/.notes/implementation-notes.html deleted file mode 100644 index d1bb62e1..00000000 --- a/.notes/implementation-notes.html +++ /dev/null @@ -1,725 +0,0 @@ - - - - - EEGPrep Core Parity Implementation Notes - - -

EEGPrep Core Parity Implementation Notes

-

Issue #213 Statistics and ICLabel Closeout Notes

-

Issue #213 closes the final Fable 5 architecture closeout phase for - statistics and ICLabel/viewprops ownership. The branch keeps public call - signatures, return values, command strings, GUI labels, and Matplotlib - dashboard layout behavior stable while moving implementations into focused - owner modules.

-

Issue #213 Design Decisions

- -

Issue #213 Closeout Mapping for Remaining #197/#205 Items

- - - - - - - - - - - - - - - - - - - -
Remaining itemDispositionRationale
pop_load_frombids.py raw-reader and montage - ownershipFixed in PR #216 / issue #212BIDS raw loading - and montage inference moved into EEG-BIDS helper modules while - pop_load_frombids stayed orchestration/history glue.
functions/statistics/_core.py mega-moduleFixed - in issue #213Implementations now live in same-name statistics - modules, with _shared.py holding only shared helpers.
plugins/ICLabel/pop_prop_extended.py mixed - ownershipFixed in issue #213Pop/history/dialog glue, - numerics/data assembly, and Matplotlib browser rendering now have - separate owner modules.
functions/guifunc/qt.py stateless renderer namespaceFixed - in PR #216 / issue #212Qt renderer helpers were split into - module-level helpers while preserving call-site compatibility and dialog - behavior.
pop_clust finite-outlier robust k-means dispatchFixed - in PR #217 / issue #210STUDY clustering behavior was consolidated - in the STUDY/time-frequency/statistics phase with dedicated tests.
pop_chanplot, measure-field maps, cached measure - axes, range masks, default plot targets, _trial_rows, factor - matching, and std_clustplot history constructionFixed - in PR #217 / issue #210STUDY measure/cache helper ownership was - consolidated as one phase because those paths share STUDY data contracts.
Time-frequency numeric-vector parsing, bootstrap/FDR helpers, - threshold-vector helpers, and empirical p-value convention auditFixed - or explicitly documented in PR #214 / issue #208 and PR #217 / issue - #210Shared parsing moved to lower-level helpers in Phase 1; - time-frequency and statistics behavior consolidation landed in Phase 3 - where numerical behavior could be tested together.
Divergent is_on(), empty-value, Python literal, - chanloc serialization, topographic channel selection, boundary-event - detection, and ConsoleEegh history mutation copiesFixed - in PR #214 / issue #208Shared low-level contracts were centralized - before later phases consumed them.
Rejection-family browser plumbing, epoched rejection scaffolds, - component activation access, ICA finalization, and clean_rawdata - channel-removal masksFixed in PR #215 / issue #209These - scientific rejection/ICA/cleaning helpers were consolidated in the phase - that could test visual rejection state and numerical side effects - together.
CLI transform/pipeline duplication, JSON detection, stale - per-module harness contracts, extension catalog split, entry-point - helpers, active-record predicate, and bundled-plugin metadataFixed - in PR #218 / issue #211CLI and extension ownership was - consolidated in one agent-facing architecture phase.
FIR helper ownership and FIR GUI band-edge/shape duplicationFixed - in PR #216 / issue #212plugins/firfilt now owns FIR - design helpers, and clean_rawdata imports downward from firfilt.
GUI extension pop-result STUDY return handling from #197/#205Non-goal - for #213The earlier tracker identified this as behavior-changing - because GUI extension pop_* functions returning STUDY state - would expand observable session behavior. No #213 work touched extension - GUI result semantics; this remains outside a mechanical closeout split - unless a future behavior issue requests it.
All other #197 findings not named aboveSuperseded by epic - #207 phase split#197 and #205 were closed as superseded by the - replacement architecture closeout epic. PRs #214, #215, #217, #218, #216, - and this #213 branch are the final accounting set for those remaining - findings.
-

Issue #213 Verification Notes

- -

Issue #212 BIDS, Qt, and FIR Ownership Notes

-

Issue #212 closes the Phase 5 architecture ownership work for BIDS - import helpers, Qt dialog rendering helpers, and FIR design helpers while - preserving standalone EEGPrep runtime behavior.

-

Issue #212 Design Decisions

- -

Issue #212 Verification Notes

- -

Issue #164 EEGLAB-Style Sphinx Documentation Notes

-

Issue #164 turns the final-epic documentation into a coherent standalone - EEGPrep manual modeled after EEGLAB's learning path but written for Python, - the Qt GUI, eegprep-console, and EEGPrep-owned packaged - resources.

-

Issue #164 Design Decisions

- -

Issue #164 Tradeoffs

- -

Issue #163 ICLabel/Viewprops Diagnostic Parity Notes

-

Issue #163 closes the Phase 6 final-epic ICLabel/viewprops rows by adding - standalone label statistics, making alternate network support explicit, and - documenting the native EEGPrep diagnostic workflow.

-

Issue #163 Design Decisions

- -

Issue #163 Verification Notes

- -

Issue #158 Final Standalone Parity Audit Notes

-

Issue #158 establishes the Phase 1 contract for epic #157. It adds an - enforceable final-epic matrix for bundled plugin depth, MATLAB object/storage - semantics, optional-toolbox workflows, and EEGLAB-style docs architecture - without porting feature behavior.

-

Issue #158 Design Decisions

- -

Issue #158 Runtime Contract

- -

Issue #158 Tradeoffs

- -

Issue #158 Verification Notes

- -

Issue #147 Correct MC Random-Symbol Fitting Notes

-

Issue #147 implements the Ramberg-Schmeiser/random-symbol fitting stack - needed by correct_mc while keeping EEGPrep runtime code independent - of the vendored EEGLAB checkout.

-

Issue #147 Design Decisions

- -

Issue #147 Tradeoffs

- -

Issue #147 Verification Notes

- -

EEGPrep Follow-up #149 STUDY Long-Tail Notes

-

Follow-up #149 closes the non-PAC Phase 6 STUDY helper rows that remained - after the core STUDY implementation. The branch implements the standalone - selection and trialinfo helpers that EEGPrep workflows can use directly, and - classifies the remaining plotting, precompute, MATLAB-runtime, or - external-toolbox rows with row-level rationale in the parity matrix.

-

Follow-up #149 Design Decisions

- -

Follow-up #149 Classifications

- -

Follow-up #149 Verification Notes

- -

EEGPrep Phase 8 Epic Closeout Notes

-

Phase 8 reviews the integrated Phases 1-7 surface for closeout evidence - rather than adding another feature area. The branch remains scoped to matrix, - docs/help, QA evidence, and concrete integration defects.

-

Phase 8 Closeout Findings

- -

Phase 8 Tradeoffs

- -

Phase 8 Verification Notes

- -

EEGPrep Phase 7 Long-Tail Core Helpers Notes

-

Phase 7 closes the remaining audit-approved helper and wrapper gaps that - were still actionable after Phases 2-6, while classifying obsolete or - toolbox-dependent MATLAB surfaces explicitly in the parity matrix.

-

Phase 7 Design Decisions

- -

Phase 7 Deviations

- -

Phase 7 Follow-Ups

- -

Phase 7 Verification Notes

- -

EEGPrep Phase 5 Unsupported User-Facing Options Notes

-

Phase 5 closes option-level gaps in existing user-facing functions while - keeping EEGPrep standalone and explicit about MATLAB-only or toolbox-dependent - behavior.

-

Phase 5 Design Decisions

- -

Phase 5 Tradeoffs

- -

Phase 5 Verification Notes

- -

EEGPrep Phase 6 STUDY And Group-Level Notes

-

Phase 6 deepens standalone STUDY workflows without adding runtime dependencies - on the vendored EEGLAB tree or pretending external LIMO behavior is available.

-

Phase 6 Design Decisions

- -

Phase 6 Deviations

- -

Phase 6 Tradeoffs

- -

Phase 6 Open Questions

- -

EEGPrep Phase 2 File Format And Channel-Location Notes

-

Design Decisions

- -

Deviations

- -

Tradeoffs

- -

Open Questions

- -

Core Parity GUI Layout Pass

- -

Final Review Fixes

- -

Follow-up #148 PAC Classification

- -

Follow-up #146 Time-Warped newtimef And Cycle Calculator Notes

-

Design Decisions

- -

Deviations And Tradeoffs

- -

Verification Notes

- -

Phase 2 clean_rawdata And FIRFilt Completion Notes

-

Design Decisions

- -

Deviations And Tradeoffs

- -

Verification Notes

- -

Phase 3 DIPFIT And Source-Localization Notes

-

Design Decisions

- -

Deviations And Tradeoffs

- -

Verification Notes

- -

Phase 4 #161 STUDY PAC, LIMO Design, And Neighbor Notes

-

Design Decisions

- -

Tradeoffs And Boundaries

- -

Verification Notes

- -

Phase 5 Large-Dataset Storage Notes

-

This phase implements EEGPrep-owned storage behavior for issue #162 without - adding runtime dependencies on the vendored EEGLAB tree or porting MATLAB - @memmapdata/@mmo overloads one-for-one.

-

Design Decisions

- -

Tradeoffs And Limitations

- -

Verification Notes

- -

Issue #175 CLI Transform Command Notes

-

Design Decisions

- -

Tradeoffs And Boundaries

- -

Verification Notes

- -

Issue #175 Agent-Friendly CLI Closeout Notes

-

Design Decisions

- -

Tradeoffs And Boundaries

- -

Verification Notes

- -

Async GUI ICA Progress Notes

-

Design Decisions

- -

Tradeoffs

- -

Issue #211 CLI And Extension Architecture Notes

-

Design Decisions

- -

Verification Notes

- - - diff --git a/docs/parity/eeglab_core_parity_matrix.json b/docs/parity/eeglab_core_parity_matrix.json index 94ff22b2..76ac63d5 100644 --- a/docs/parity/eeglab_core_parity_matrix.json +++ b/docs/parity/eeglab_core_parity_matrix.json @@ -1,7 +1,6 @@ { "metadata": { "schema_version": 1, - "source_audit": ".notes/eeglab-migration-gap-audit.md", "source_issue": "https://github.com/sccn/eegprep/issues/132", "parent_epic": "https://github.com/sccn/eegprep/issues/131", "scope": { diff --git a/docs/parity/eeglab_final_parity_matrix.json b/docs/parity/eeglab_final_parity_matrix.json index b55e5401..19a25e57 100644 --- a/docs/parity/eeglab_final_parity_matrix.json +++ b/docs/parity/eeglab_final_parity_matrix.json @@ -1,7 +1,6 @@ { "metadata": { "schema_version": 1, - "source_audit": ".notes/eeglab-final-parity-audit.md", "source_issue": "https://github.com/sccn/eegprep/issues/158", "parent_epic": "https://github.com/sccn/eegprep/issues/157", "scope": { diff --git a/docs/source/development.rst b/docs/source/development.rst index 5740ced4..7f28262f 100644 --- a/docs/source/development.rst +++ b/docs/source/development.rst @@ -209,9 +209,9 @@ EEGLAB Core Parity Matrix The Phase 1 core parity epic uses a committed machine-readable matrix at ``docs/parity/eeglab_core_parity_matrix.json``. The matrix classifies the -EEGLAB public and semi-public functions in the first seven migration-gap audit -categories from ``.notes/eeglab-migration-gap-audit.md``. It is a work -contract for later phase agents, not package runtime data. +EEGLAB public and semi-public functions in the scope categories recorded in +its metadata and source issue. It is a work contract for later phase agents, +not package runtime data. Rows use these statuses: @@ -299,11 +299,11 @@ Validate the final matrix with: uv run --no-sync python -m tools.eeglab_final_parity_matrix --json -The docs architecture for the final epic is recorded in the matrix metadata and -in ``.notes/eeglab-final-parity-audit.md``. It should be useful to EEG -researchers first: describe EEGPrep's standalone Python package, Qt GUI, and -``eegprep-console`` behavior accurately, and use EEGLAB comparisons only where -they help users migrate or understand familiar concepts. +The docs architecture for the final epic is recorded directly in the matrix +metadata and its source issue. It should be useful to EEG researchers first: +describe EEGPrep's standalone Python package, Qt GUI, and ``eegprep-console`` +behavior accurately, and use EEGLAB comparisons only where they help users +migrate or understand familiar concepts. Building Documentation ====================== diff --git a/src/eegprep/resources/help/pop_preclust.md b/src/eegprep/resources/help/pop_preclust.md index 9d674dc1..0d8a53d9 100644 --- a/src/eegprep/resources/help/pop_preclust.md +++ b/src/eegprep/resources/help/pop_preclust.md @@ -13,8 +13,8 @@ STUDY, ALLEEG, com = pop_preclust( ``` Scalp-map features are read from loaded ICA maps. ERP, spectrum, ERSP, and ITC -features must already be present under -`STUDY["etc"]["eegprep"]["component_measures"]`, the Phase 5b component-measure -contract documented in `.notes/implementation-notes.html`. +features must already be cached on the parent `STUDY["cluster"][0]` entry, +normally by calling `pop_precomp(STUDY, ALLEEG, "components", ...)` for the +requested measures. -See also: STD_PRECLUST, POP_CLUST, POP_CLUSTEDIT +See also: POP_PRECOMP, STD_PRECLUST, POP_CLUST, POP_CLUSTEDIT diff --git a/tests/test_guifunc_pophelp_chansel.py b/tests/test_guifunc_pophelp_chansel.py index 318499f2..cc12d8eb 100644 --- a/tests/test_guifunc_pophelp_chansel.py +++ b/tests/test_guifunc_pophelp_chansel.py @@ -60,6 +60,12 @@ def test_phase_study_pop_help_resources_are_packaged(self): self.assertIn("resources/help", Path(source_path).as_posix()) self.assertTrue(source_path.endswith(f"{target}.md")) + def test_packaged_help_does_not_reference_local_notes(self): + text, _ = pophelp_text("pop_preclust") + + self.assertNotIn(".notes/", text) + self.assertIn("pop_precomp", text) + def test_dialog_help_targets_have_packaged_resources(self): interp_eeg = {"data": [], "trials": 1, "chanlocs": [], "chaninfo": {}, "epoch": []} specs = (pop_reref_dialog_spec(), pop_interp_dialog_spec(interp_eeg))