Hot-reload .emanoteignore mid-session (closes #739)#741
Conversation
Each layer's .emanoteignore was read once at process start; editing the file during `emanote run` required restarting the binary to take effect. The seam noted in Source/Dynamic.hs is removed by moving per-layer ignore filtering out of unionmount (which only sees the fixed universal patterns now) and into the streaming handler in Emanote.Source.Dynamic: per-layer patterns live in a TVar that the fsnotify pipeline updates in place when the .emanoteignore file itself changes, the handler walks the model to evict newly-ignored notes, and walks a tracked set of previously-suppressed events to re-include files whose patterns were removed. New 'IgnoreFile' FileType claims .emanoteignore as a first-class source so unionmount delivers its events through the same pipeline as other source files. The static-file index branch in Source.Patch skips it; the patchModel branch is a no-op because Source.Dynamic has already applied the effect by the time patchModel sees the event. Covered by two new live @Hot-Reload cucumber scenarios in tests/features/emanoteignore.feature exercising both directions (pattern added → note hidden, pattern removed → note re-appears).
Sub-tree .emanoteignore events (e.g. sub/.emanoteignore) carry the same R.IgnoreFile tag as the layer-root file but configure no patterns. Skip the disk reload when a batch contains only sub-tree events; previously the predicate was exported but had no caller. Closes hickey finding #5 / lowy finding #2.
…noreFileChanges LuaFilter hot-reload is implemented inside Patch.patchModel; the .emanoteignore hot-reload runs *upstream* of patchModel, inside Source.Dynamic. A reader could miss the asymmetry without a pointer at each end — add explicit cross-references to keep the structure discoverable. Closes lowy finding #4.
Matches the codebase convention (ModelT, Note, StaticFile, EmanoteConfig) of accessing internal record state through optics rather than raw underscore-prefix selectors. Closes lowy finding #3.
applyFiltered and reEmitModified both ran their own NE.filter + length comparison + nonEmpty branch to decide whether to dispatch Refresh, Refresh-with-trimmed-overlays, or Delete. Lift the pure decision into 'Emanote.Source.Ignore.classifyOverlays' returning an 'OverlayOutcome' so both call sites read as: classify → branch → dispatch. The bookkeeping (recordModified/forgetModified) stays at the call site where it belongs. Closes hickey finding #4.
…lice Previously _isPatterns and _isModifiedEvents were two TVars whose invariants (entry absent from set ↔ visible in model; entry present ↔ suppressed) were held by hand at three sites. Combine them into a single 'IgnoreSlice' record behind one TVar and route every read through 'snapshotSlice' / every write through 'updateSlice'. The single-transaction property gives a future invariant linking the two fields one place to live, and 'recordModified' / 'forgetModified' now delegate to the slice helper instead of poking a raw TVar. Closes hickey finding #3.
applyFiltered, applyOne, handleIgnoreFileChanges, evictNewlyIgnored, and reEmitModified each threaded the same four invariants positionally (layers, noteFn, storkIndex, ignoreState). Bundle them into a single HandlerCtx so a new helper that needs (say) storkIndex alongside the ignore state doesn't have to grow another positional argument. Closes hickey finding #6.
evictNewlyIgnored only walked _modelNotes, leaving static files (images, PDFs, source-code embeds, ...) serving when their path matched a freshly-added pattern. Add _staticFileSource :: Maybe (Loc, FilePath) to StaticFile, populate it from the top overlay in Patch.insertStaticFile, and walk _modelStaticFiles alongside notes in evictNewlyIgnored. The hot-reload coverage for YAML data and Heist templates remains a known limitation — neither carries per-layer source metadata in the model today. Documented in docs/guide/emanoteignore.md. Closes hickey finding #2 / lowy finding #1.
…o their guides The hot-reload caveat for YAML data and Heist templates now points at [[yaml-config]] and [[html-template]] so a reader following the limitation has the surrounding context one click away.
evictNewlyIgnored unconditionally overwrote any existing ledger entry with a single-element ((loc, lfp) :| []) overlay drawn from _noteSource / _staticFileSource. If a previous OverlayPartial had stashed the full multi-layer overlay (e.g. [(Layer1, foo.md), (Layer2, foo.md)]), that history would be silently lost — a subsequent pattern relaxation would resurrect the file from the wrong layer in multi-layer setups. Add 'recordModifiedIfAbsent' and route both note and static-file eviction through it. The original 'recordModified' (overwrite-wins) stays the right helper for 'applyFiltered', where the new event from unionmount carries the freshest overlay.
…leType The handwritten isMdRoute case-split duplicates the 'HasExt' class's 'fileType' method, which already resolves to the correct 'R.LMLType R.Md' / 'R.LMLType R.Org' per its instance. Route via 'withLmlRoute' so the type-class machinery handles the choice.
snapshotSlice / updateSlice / changeEntries / applyOne / mountedPath each had a one-line Haddock that restated the identifier name. Per the codebase convention (default to no comments; keep only non-obvious why), delete them. The two record-helpers (recordModified vs recordModifiedIfAbsent) keep shortened comments naming the policy choice — that's the non-obvious distinction.
…e with pointer The 14-line comment above the R.IgnoreFile branch in Patch.hs repeated prose that already lives in the haddock for handleIgnoreFileChanges in Dynamic.hs. Replace with a 3-line cross-reference; Dynamic.hs is the single source of truth for the LuaFilter-vs-IgnoreFile asymmetry.
…erns empty Every fsnotify event during a live-reload session runs through classifyOverlays. For the common case where no layer has any per-layer patterns, the NE.filter walk + per-overlay Map lookups are all no-op work. Add a fast path that returns OverlayKept directly when the pattern map is empty.
…patterns changed evictNewlyIgnored and reEmitModified previously walked the full _modelNotes and _modelStaticFiles on every .emanoteignore edit. For a 10k-note notebook each save was O(N), even though a typical edit touches only one layer's pattern set. Diff old vs new patterns per Loc once, then filter every walk by Set membership before applying the heavier isLayerPathIgnored / classifyOverlays predicates. Correctness: if a layer's patterns didn't move, no file in that layer can have flipped sides.
Three step definitions (and one new one from this PR) repeated the
same path.join + mkdirSync({recursive: true}) + writeFileSync trio.
Lift it into support/fixture.ts next to stagedFixtureDir; both
step files reuse it. A future step that writes a new fixture file
gets the right ceremony for free.
…steps
Three step definitions ('URL X contains Y within Ns', 'URL X stops
containing Y within Ns', 'I wait for X to contain Y') used the same
request-inspect-sleep-retry loop. Lift it into tests/support/poll.ts
behind a typed predicate + onTimeout pair so each step shrinks to a
4-line wrapper; the page-DOM 'article body contains … within Ns'
step stays put since it uses Playwright's waitForFunction.
filepattern's '**' matches the empty path, so the master pattern '**/.*/**' silently matched a root-level dotfile like '.emanoteignore' (via ** = "", .* = ".emanoteignore", trailing ** = ""). The previous ignorePatterns list passed this to unionmount as a per-source ignore; unionmount's fsnotify watcher then filtered every .emanoteignore modify event before it could reach handleIgnoreFileChanges, breaking the entire hot-reload pipeline this PR was meant to deliver. Tighten the pattern to '**/.*/**/*' so the trailing component is required. Dotfile-directory CONTENTS (.git/HEAD, .vscode/settings.json, .git/objects/00/abc) still match; root-level standalone dotfiles (.emanoteignore, .gitignore, .envrc, …) no longer do. Behavior change: root-level dotfiles that the universal pattern was incidentally filtering — .gitignore, .envrc, .editorconfig, etc. — are now exposed as AnyExt static files unless the user adds them to their own .emanoteignore. Most users have nothing sensitive in those files; users who do can list them in .emanoteignore. Verified manually: with this fix the two new hot-reload scenarios in tests/features/emanoteignore.feature flip the matching note's visibility in well under the 10-second polling budget.
The hot-reload entry now flags that the universal pattern change to let .emanoteignore through also stops swallowing other root-level standalone dotfiles (.gitignore, .envrc, .editorconfig, …). Dotfile *directories* (.git/, .vscode/) are unchanged.
Hickey/Lowy Analysis
Hickey rationaleThe diff initially folded both filtering decisions and ledger bookkeeping into single match arms of The two TVars (
Session-stable handler parameters ( Lowy rationaleThe volatility being encapsulated is "what counts as a layer-root configuration file." Master left it overloaded with "what kinds of source files unionmount routes through the typed pipeline" — The
|
EvidenceThree WebSocket-morphed states captured in a single browser tab at State 1 — Baseline.
State 2 — Pattern removed. The
State 3 — Pattern re-added. The line is written back to
|
|
| Step | Status | Duration | Verification |
|---|---|---|---|
| sync | ✓ | 0s | forge=github, branch=hot-reload-emanoteignore |
| research | ✓ | 12m 10s | design settled: emanote-only TVar+ledger approach |
| branch | ✓ | 5s | feature branch hot-reload-emanoteignore |
| implement | ✓ | 15m 6s | IgnoreFile FileType; classifyOverlays; TVar-driven hot-reload; 2 cucumber scenarios |
| check | ✓ | 14s | cabal build all clean |
| docs | ✓ | 1m 17s | CHANGELOG bullet + emanoteignore.md Hot reload section |
| fmt | ✓ | 49s | hlint + fourmolu + cabal-fmt + nixpkgs-fmt all green |
| commit | ✓ | 24s | primary feature commit pushed |
| hickey+lowy | ✓ | 34m 50s | 10 findings: 1 No-op (empirically incorrect), 7 fix commits, 2 cross-lens dups |
| police | ✓ | 28m 9s | 1 rule fix + 1 fact-check + 7 elegance fixes (9 commits) |
| test | ✓ | 39m 56s | e2e-live 81/81 incl. 2 new scenarios; static 60+21 skipped; unit 125/125 |
| create-pr | ✓ | 1m 32s | draft PR #741 + hickey+lowy analysis comment |
| ci | ✓ | 1m 54s | vira ci both signoffs; e2e-morph 81/81 |
| evidence | ✓ | 2m 22s | three before/middle/after screenshots |
| Total | 2h 19m 22s |
Slowest step: test (39m 56s) — dominated by an e2e-live run on a binary built before the fsnotify root-cause was found.
Optimization suggestions
- Verify the live happy path manually before running the full e2e suite.
testburned ~25 minutes on a green-looking build that the new@hot-reloadscenarios were silently failing against. A 60-secondcp fixture → emanote run → edit .emanoteignore → curlwould have surfaced the**/.*/**filter-pattern bug before the slow Cucumber + Playwright pass. - Pre-test filepattern semantics for any pattern you change. The whole second hour of this run was triggered by
**/.*/**accidentally matching root-level dotfiles (filepattern's**matches the empty path). A 3-line?==smoke test would have caught this at design time, in theresearchstep. - Combine related police fixes that touch the same file. The seven elegance commits each round-tripped through
nix develop -c cabal build emanote+just fmt(~30s each). Batching the trivial in-file ones (e.g.lmlRouteFileType+ trim-Haddock + collapse-TVars all touchedSource/Dynamic.hs) would shave a few minutes off without losing the per-finding commit message clarity. - Re-use
--from polishor--from ci-onlyon retries. The pattern fix needed only a rebuild + e2e replay, but it walked the full workflow from check onwards. The CLI already supports skipping to the relevant entry point.
Workflow completed at 2026-05-17.
1 participant
emanote runnow picks up edits to a layer's.emanoteignorewithout a restart. Adding a pattern evicts matching notes and static files from the model and the sidebar; removing a pattern brings the previously-hidden ones back. Closes #739.The seam the issue described — unionmount accepts ignore patterns as a static
Map source [FilePattern]— is sidestepped by moving per-layer filtering out of unionmount entirely. unionmount keeps only the universal patterns (dotfile-dir contents, vim backups, the reserved-/); per-layer.emanoteignorecontent lives in aTVarthat.emanoteignorechange events update in place. The streaming handler reads that TVar on every event and the dedicated reload path walks_modelNotes+_modelStaticFileswhenever the pattern set actually moves.Flow
Emanote.Route.Extgains a newIgnoreFileconstructor so unionmount surfaces.emanoteignoreevents through the same tagged pipeline as.md/.lua/.yaml. TheR.IgnoreFilebranch ofpatchModelis a no-op — by the time an event reaches it,handleIgnoreFileChangeshas already reshaped the model upstream.Coverage
Two new
@live @hot-reloadCucumber scenarios intests/features/emanoteignore.featurecover both directions:.emanoteignoreexcludessecret-ignored.md/secret-ignored.htmlcontains marker within 10s.emanoteignoresecret-ignored.mdBehavior change
Notable limitations (documented in
docs/guide/emanoteignore.md)YAML data files (the metadata cascade) and Heist
.tpltemplates don't carry per-layer source metadata in the model yet, so adding a pattern matching one mid-session leaves the resolved cascade / template loaded until restart. Removing such a pattern is unaffected because those files were never filtered out in the first place.Try it locally
Generated by
/doon Claude Code (modelclaude-opus-4-7).